python-for-android 2019.08.09 released: running under Python 2 no longer supported

python-for-android is a packaging tool for Python apps on Android. You can create your own Python distribution including the modules and dependencies you want, and bundle it in an APK along with your own code.


python-for-android 2019.08.09 has just been released! I haven’t been making a blog post for every new version now that we’ve switched to a monthly releases, but this one includes an especially major change: python-for-android no longer runs under Python 2. If this affects you, simply run python-for-android (or buildozer) under Python 3.6 or newer, they should run exactly the same as ever.

Note that you can still target Python 2 on Android, simply add python2 to your requirements as you would normally. However, this won’t last forever, support for Python 2 will probably be totally removed early in 2020.

Other changes in this release include further testing improvements, significantly improved documentation on developing with python-for-android, and some preparations for further big changes in the future. See the release note for full details.

A delayed resize layout in Kivy

A user on the Kivy Discord just raised the question of how to delay widget updates during resize events. The problem was that the widgets did some heavy processing (generating matplotlib graphs) that would be very slow if called for every tiny update during a larger resize event.

This is a good opportunity to return to the flexibility of Kivy layouts. It’s very easy to add some simple behaviour that delays updates until a short period has passed without the size changing. Here’s a quick implementation I threw together:

from kivy.uix.anchorlayout import AnchorLayout
from kivy.clock import Clock
from kivy.properties import ObjectProperty, NumericProperty

from functools import partial

class DelayedResizeLayout(AnchorLayout):

    do_layout_event = ObjectProperty(None, allownone=True)

    layout_delay_s = NumericProperty(0.2)

    def do_layout(self, *args, **kwargs):
        if self.do_layout_event is not None:
            self.do_layout_event.cancel()
        real_do_layout = super().do_layout
        self.do_layout_event = Clock.schedule_once(
            lambda dt: real_do_layout(*args, **kwargs),
            self.layout_delay_s)

This layout could be used as the root widget of a whole application, to delay the resizing of all the content, or somewhere within the app to delay only a small part of it.

And a simple example:

from kivy.uix.button import Button
from kivy.base import runTouchApp

button = Button(text='example button')
layout = DelayedResizeLayout()
layout.add_widget(button)
runTouchApp(layout)

Widget interactions between Python and Kv

One of the biggest Kivy confusions I see is how different widgets can access one another when they’re in different parts of the application. Fortunately, it’s generally straightforward to do so. This post gives examples of methods you might use in different situations.

The emphasis here is on what you can do, not what you should. If you aren’t sure which way is best in a given situation, go ahead and choose what seems best at the time, but don’t be afraid to revisit it later.

How can a widget access its parent?

Use the parent property of the widget:

child = Widget()
parent = Widget()

assert child.parent is None  # child widget isn't added to any parent

parent.add_widget(child)

assert child.parent == parent

How can a widget access its children?

Use the children property. This is a list containing all the children you added.

child = Widget()
parent = Widget()

assert len(parent.children) == 0  # no children added to parent

parent.add_widget(child)

print(parent.children)  # [<kivy.uix.widget.Widget object at 0x???>]
assert child in parent.children

Note: The children list is actually backwards, i.e. the last widget you add will by default be the first in the list. This is a bit surprising, and only really happens for backwards compatibility.

How can a widget in Python access children from its kv rule?

Option 1: ids

You can give the widgets in your kv rule ids to access them from Python.

# main.py

from kivy.uix.boxlayout import BoxLayout
from kivy.uix.label import Label
from kivy.app import App

class KvRuleWidget(BoxLayout):
    def on_touch_down(self, touch):
        print('We can get references to all the children using the ids dict.')

        # syntax is `self.ids.<id_text>`

        assert self.ids.middle in self.children
        assert self.ids.bottom in self.children

        # widgets can be accessed from deep in the kv rule
        assert self.ids.top_left not in self.children
        assert self.ids.top_right not in self.children

class ExampleApp(App):
    def build(self):
        return KvRuleWidget()
# example.kv

<KvRuleWidget>:
    orientation: 'vertical'
    BoxLayout:
        orientation: 'horizontal'
        Label:
            id: top_left
            text: 'top left'
        Label:
            id: top_right
            text: 'top right'
    Label:
        id: middle
        text: 'middle'
    Label:
        id: bottom
        text: 'bottom'

Note: You cannot set up widget ids from Python code, if you write e.g. w = Widget(id='some_name') this will not crash but the id will not be available in any ids dictionary.

Option 2: properties

You can use Kivy properties to pass around references to widgets.

# main.py

from kivy.uix.boxlayout import BoxLayout
from kivy.uix.label import Label
from kivy.app import App
from kivy.properties import ObjectProperty

class KvRuleWidget(BoxLayout):
    top_right_label = ObjectProperty()

    def on_touch_down(self, touch):
        print('The top right label is {}'.format(self.top_right_label))

class ExampleApp(App):
    def build(self):
        return KvRuleWidget()
# example.kv

<KvRuleWidget>:
    orientation: 'vertical'
    top_right_label: top_right  # note that we used an id to set the property
    BoxLayout:
        orientation: 'horizontal'
        Label:
            id: top_right
            text: 'top left'
        Label:
            text: 'top right'
    Label:
        text: 'middle'
    Label:
        text: 'bottom'

Option 3: The parent and children properties

It is possible to walk through the widget tree using the parent and children properties.

This is usually a bad idea and is prone to breakage if the structure of the widget tree changes. However, it’s still possible.

# main.py

from kivy.uix.boxlayout import BoxLayout
from kivy.uix.label import Label
from kivy.app import App
from kivy.properties import ObjectProperty

class KvRuleWidget(BoxLayout):
    def on_touch_down(self, touch):

        # get a reference to the top right label only by walking through the widget tree
        top_right_label = self.children[-1].children[0]

        print('The top right label is {}'.format(self.top_right_label))

class ExampleApp(App):
    def build(self):
        return KvRuleWidget()
# example.kv

# note: this time there are no ids at all
<KvRuleWidget>:
    orientation: 'vertical'
    BoxLayout:
        orientation: 'horizontal'
        Label:
            text: 'top left'
        Label:
            text: 'top right'
    Label:
        text: 'middle'
    Label:
        text: 'bottom'

How can a widget in Kv access children defined in Python?

Sometimes you might have some children defined via a Kv rule, and others created dynamically in Python. You can access the Python widgets in kv by saving references to them in Kivy properties:

# main.py

from kivy.uix.boxlayout import BoxLayout
from kivy.uix.label import Label
from kivy.app import App
from kivy.properties import ObjectProperty

class KvRuleWidget(BoxLayout):
    label_created_in_python = ObjectProperty()

    def __init__(self, **kwargs):
        super().__init__(**kwargs)

        # add a widget from python code
        label = Label(text='label created in Python')
        self.add_widget(label)
        self.label_created_in_python = label  # save a reference

class ExampleApp(App):
    def build(self):
        return KvRuleWidget()
# example.kv

<KvRuleWidget>:
    orientation: 'vertical'
    Label:
        text: 'label created in Kv'
    Label:
        text: 'the label created in Python has text "{}"'.format(root.label_created_in_python.text)

How can a widget defined in a kv rule access a widget defined in another kv rule?

Sometimes you might have two widgets in very different places that need to talk to one another somehow. Usually the best way to achieve this is to consider how they are related to one another, and pass information between them via their common relations.

Also see the next Section for how to access any widget from anywhere, without worrying about how the widgets are related. However, that usually isn’t such a good choice in the long run.

The following example is deliberately very simple, but the same principles can be used to link together widgets across your whole program using references passed around where the kv rules meet.

# main.py

from kivy.uix.boxlayout import BoxLayout
from kivy.uix.button import Button
from kivy.uix.label import Label
from kivy.app import App
from kivy.properties import ObjectProperty

class IncrementCounterButton(Button):
    counter = NumericProperty(0)
    def on_press(self):
        self.counter += 1

class CounterLabel(Label):
    counter = NumericProperty(0)

class RootWidget(BoxLayout):
    pass

class ExampleApp(App):
    def build(self):
        return RootWidget()
# example.kv

<IncrementCounterButton>:
    text: 'press me'

<CounterLabel>:
    text: 'the counter value is {}'.format(app.counter)  # `app` in kv is equivalent to `App.get_running_app()` in Python

<RootWidget>:
    orientation: 'vertical'
    CounterLabel:
        counter: button.counter  # this means the CounterLabel's counter will always match the button's counter
    IncrementCounterButton:
        id: button

How can any widget access any other widget from anywhere?

Sometimes you really do want widgets to interact with one another without any good relationship between them. You can do this in a convenient way by using a Kivy property in the App class.

Note: This is notionally similar to using a global variable, and is often bad practice for all the same reasons.

The following example is quite contrived to keep it simple. In this case you could probably think of a better way to do the same thing, perhaps using the methods from the previous Sections.

# main.py

from kivy.uix.boxlayout import BoxLayout
from kivy.uix.button import Button
from kivy.uix.label import Label
from kivy.app import App
from kivy.properties import ObjectProperty

class IncrementCounterButton(Button):
    def on_press(self):
        # You can always access your App class from Python as follows:
        App.get_running_app().counter += 1

class CounterLabel(Label):
    pass

class ExampleApp(App):
    counter = NumericProperty(0)

    def build(self):
        boxlayout = BoxLayout(orientation='vertical')
        label = CounterLabel()
        button = IncrementCounterButton()

        boxlayout.add_widget(label)
        boxlayout.add_widget(button)

        return boxlayout
# example.kv

<IncrementCounterButton>:
    text: 'press me'

<CounterLabel>:
    text: 'the counter value is {}'.format(app.counter)  # `app` in kv is equivalent to `App.get_running_app()` in Python

An update on python-for-android: v2019.06.06 released and future plans

python-for-android is a packaging tool for Python apps on Android. You can create your own Python distribution including the modules and dependencies you want, and bundle it in an APK along with your own code.


python-for-android 2019.06.06 has just been released! This release contains 198 commits from 31 different contributors. Many thanks to everyone involved.

Major changes in this release include:

  • Added support for running your setup.py when packaging your app, enabling your code to be installed as a module in the on-device Python environment. This also makes it easy to build Cython or other compiled components.
  • Added some tooling for requesting and checking for app permissions at runtime.
  • Added initial support for matplotlib.
  • Updated many recipes, and especially the SDL2 backend components, for improved performance and stability on Android.
  • Ongoing improvements to our test coverage and infrastructure.
  • Removed a significant amount of dead code relating to long-deprecated build configurations, and unified other parts of the build to reduce duplication of effort between bootstraps.
  • Updated the release model to target regular, smaller releases.

Of course there have also been a wide range of bugfixes and improvements. See the release notes for a full changelog.

Release model

In this release we’ve transitioned to a calendar-based version system. Future releases will continue to be of the form YYYY.MM.DD. We’re initially targeting one release every four weeks. This scheme represents how python-for-android is normally best used: many changes are driven by updates in the Android ecosystem and build toolchain, and in practice it’s usually best to be working from the most recent possible python-for-android release.

This has been made possible by the hard work of various contributors, setting up and continuing to improve python-for-android’s suite of tests and continuous integration services. In the past we haven’t done a great job of keeping up releases alongside major improvements, but this should now be much more straightforward.

If you’re using buildozer then this doesn’t directly affect you, as buildozer was already configured to use a more recent python-for-android version. Buildozer will now automatically transition to use the latest release, represented by the master branch in the python-for-android git repository.

Future work

We’ve had some questions about Google’s plan to require 64-bit support for apps on Google Play, starting in August 2019. In fact python-for-android already supports this, just pass the required architecture as an argument to the build:

p4a apk --arch=arm64-v8a

That said, we’re working to improve our testing and documentation around 64-bit builds, to make sure everything works as expected.

We also don’t currently support multi-architecture builds in a single output APK. This should be technically possible, but hasn’t ever been a focus because it would significantly increase the APK size. We may revisit this, but in the meantime you can upload one APK of each type to Google Play and python-for-android will automatically handle versioning different architectures correctly (i.e. arm64-v8a is an ‘upgrade’ for devices that support it, so that APK will be preferred).

The other current focus is on improving our test infrastructure, especially increasing test coverage and automation. This should further increase the ease of making regular releases, and our confidence that everything continues to work correctly!

These points are just general goals for python-for-android. There is always other maintenance work to do, and contributions of all types are always welcome.

Running buildozer in a virtual machine

This guide describes how to turn your Kivy/Python app into an APK, by running the buildozer build tool in a virtual machine. This is not the only way to run buildozer, it can work natively on Linux or MacOS or be run from the Windows Subsystem for Linux. See the Kivy documentation for more general instructions.

Creating a Virtual Machine

I’ll be using VirtualBox. Other virtualisation software should also work, but you’ll need to adapt the specific instructions.

We also need a target OS. I recommend Lubuntu 18.04, available here (or direct download link). Lubuntu is a light weight Ubuntu variant. You can also use a different distro if you like, but may need to adapt the later instructions.

Once you have downloaded the Lubuntu iso file, start VirtualBox and press New to create a new virtual machine. You’ll see a dialog like the following:

New VM dialog

Fill in the other options as shown in the image. It’s fine to set a larger memory size if you have enough available, or less may also work fine. Then press Create to continue.

Disk image creation dialog

You now need to select a file size for your virtual hard disk. 15 GB should be sufficient, but it’s safest to double that. Leave the other options unchanged and choose Create to continue.

Now, select your new VM and click Start in the main VirtualBox GUI. You should be prompted to select a virtual optical disk to boot from:

Choose iso to boot

Navigate to your Lubuntu 18.04 iso downloaded earlier, as shown, and press Start to continue. The first screen you see should look something like the following:

Select language

Select your language to see the boot menu:

Choose boot option

Choose the second option, "Install Lubuntu". It doesn’t matter if you accidentally press enter to "Try Lubuntu without installing", in this case there should be an Install Lubuntu application on the desktop that you can click to continue the install process.

You’ll be shown a series of dialogs to help prepare the install process. Clicking through with the defaults is fine, or select other options if you prefer.

The fourth screen will ask what kind of install to use, as shown:

Choose install type

The options shown above should be the defaults, and are what you want to use, so go ahead and continue.

Choose partitioning options

Next, select "Erase disk and install Lubuntu". Note that this is only erasing the (emtpy) virtual disk image created earlier, it won’t affect your host operating system.

Click through again, and you’ll eventually reach the user creation screen. It doesn’t matter what your username is, I used kivyuser:

User creation dialog

Click "Continue" to finally start the install. You’ll be asked a few more questions, but eventually will just have to wait for the installation to complete. This shouldn’t take too long. You’ll be prompted to "Restart Now", which you should go ahead and do.

Restart Now screen

If you have any issues with the virtual machine failing to reboot, go ahead and select Machine > Reset in the VirtualBox menu, it doesn’t matter how you do it as long as the machine is reset. If all goes well, Lubuntu should now automatically boot to a login screen - congratulations, your virtual machine is ready to use! Enter your username and password, and proceed to the next section of instructions.

Login screen

Setting up buildozer

We can now go ahead and set up buildozer ready to build your app. Open an LXTerminal as below:

LXTerminal location in menu

We now have to run a few commands to install everything buildozer needs to run. Run the following command to do so, and enter your user’s password if prompted:

sudo apt-get install python3-pip openjdk-8-jdk autoconf libtool python3-venv

That should give us everything we need for a basic app, so we can go ahead and install buildozer:

.. code-block:: sh
python3 -m venv buildozer-env source buildozer-env/bin/activate pip install buildozer cython

Note that we installed cython as well, this is also required for building the APK.

You only have to create the virtual environment once, but if you reboot the virtual machine you’ll need to run source buildozer-env/bin/activate again. See the Python documentation for more details.

The final step before running buildozer is to have your app ready in the virtual machine. You can access a folder in your host machine using VirtualBox shared folders (in the Devices > Shared Folders menu), but I won’t cover the details here. Note though that if you do this you must copy the folder contents to a different folder within the virtual machine, the buidozer process will not work if run within a shared folder.

In the following instructions I’ll assume you’ve created a folder named app_dir and placed a main.py file inside it containing your application code. Navigate to this folder in the terminal (cd app_dir) and run:

.. code-block:: sh
buildozer init

This will create a buildozer.spec file alongside your main.py:

Creating buildozer.spec

Edit the buildozer.spec to set any options you like. In this example I’ve changed only the title and pacakge.name options:

Editing buildozer.spec

I recommend changing very little for this first build, to make sure everything works. It won’t cause any problems if you edit the buildozer.spec again later.

Running buildozer

We’re now ready to actually build the app into an APK file. Start the process with:

.. code-block:: sh
buildozer -v android debug

The -v option asks for verbose output. This is recommended so that you can keep an eye on what’s happening - the details aren’t too important, but you should be able to see that the process never stops in one place for too long.

Buildozer will now download the Android tools it needs. This may take a while.

At some point you’ll be asked to accept the Android SDK license agreement, which is printed for you as in the following image:

SDK license agreement

At this point, press "y" and then enter to accept the agreement (or abort the process if you don’t agree). This is necessary even if you don’t see any text asking you to do so, due to a bug in buildozer (fixed in the next release).

After downloading everything it needs, buildozer will work through the build process compiling and packaging each of the components for your app. This may take a while, but as long as it doesn’t crash then everything is fine. Future builds will be much faster unless you change the build options, as only the contents of your app itself will need re-packaging.

Eventually the build will complete, you’ll see a screen like the following:

Build complete

That’s it, you’re done! You can find the finished APK in the bin directory, as noted in the final message buildozer prints.

Kivy Hackathon and FOSDEM 2019

For the last 3 days several of the Kivy Core Developers gathered in Brussels, Belgium for the first ever core developer hackathon. Not only is this the first time we’ve gathered to work on Kivy framework issues, but for most of us the first time we’ve ever met - even after years of collaboration!

From left to right: Peter Badida (KeyWeeUsr), Gabriel Pettier (tshirtman), Mathieu Virbel (tito), Alexander Taylor (inclement) and Andre Miras (AndreMiras)

From left to right the attendees are Peter Badida (KeyWeeUsr), Gabriel Pettier (tshirtman), Mathieu Virbel (tito), Alexander Taylor (inclement) and Andre Miras (AndreMiras).

We used this opportunity to make amazing progress on a range of Kivy issues. Most especially:

  • 55 closed issues and 36 merged pull requests across all Kivy projects, specifically targeting long standing bugs and the most important current issues.
  • A full update of buildozer to work well with the latest SDK, NDK and other dependencies for Android development. New release to follow!
  • Rapid progress on ncis, a new remote inspector/debugger for all types of Python application.
  • Many improvements to the python-for-android continuous integration, and some major bugfixes for stability with Python 3 and the latest toolchain. Preliminary 0.7.0 release already available via PyPI and buildozer, more to follow.
  • Extensive cleanup and improvements for PyJNIus.
  • A further ~170 closed python-for-android issues and ~110 buildozer issues, as a general clean up of stale and outdated reports to focus on the ones that really matter.

For the next two days, the same team will be attending FOSDEM 2019. Let us know by discord or other communication if you’d like to say hello!

Cached and templated files in python-for-android builds

I’ve more than once seen people confused by how python-for-android constructs an Android project that can be compiled into an APK. Since p4a uses various cached and templated files, it’s easy to get confused trying to edit things only to find your changes are overwritten when you run p4a again.

Here’s a quick summary of what’s what.

Bootstraps

The core project files are all in the bootstraps directory. A bootstrap is basically an Android project, containing java sources, JNI stuff to be compiled, and project management files like the AndroidManifest.xml and build.gradle.

Note: Some of these files are built from templates, e.g. in the main SDL2 bootstrap you can find build.tmpl.gradle, AndroidManifest.tmpl.xml etc. in the templates subdirectory.

If you edit the code in these bootstrap directories, the changes will always take effect if you rebuild your whole project, but might not affect anything if you repeat a previous build. This is due to caching of previous build components. You can always guarantee that everything is cleared to be rebuilt using p4a clean dists builds (or there are other tricks if you look into it).

Build directories

Most of the individual python-for-android recipes are built in separate build directories. The location of these depends on the OS: on Linux the default is ~/.local/share/python-for-android, especially ~/.local/share/python-for-android/other_builds. When you build a recipe, it is copied here for the build to be run. These folders also cache builds, so if you e.g. build two different projects using python3 then the same build directory is used both times, reducing time spent compiling.

You can edit code in ~/.local/share/python-for-android/other_builds, but it isn’t recommended unless you have a good idea what python-for-android will overwrite or cache on each run. It is, however, sometimes useful during development.

Dists

Dists are p4a’s fully compiled Android projects, including the bundled output of all the different requirements specified by the user. On Linux, their default location is ~/.local/share/python-for-android/dists.

Every time you build an APK using p4a apk ..., this essentially runs gradle from an appropriate dist directory (i.e. the one with the recipes you wanted). At this point, nothing new is built or copied from the build directories, so changes you make to p4a’s recipes or build directories have no effect on the output. However, templated files such as AndroidManifest.tmpl.xml are rebuilt every time. That means that you must edit the templates themselves if you want your changes to make it to the APK.

You can always run p4a clean dists to delete the existing Android projects. Next time you run p4a apk ..., a new dist will be created.

Buildozer

When using buildozer, everything works basically the same except that the build and dist directories can be found in $PROJECT_DIR/.buildozer/android/platform/build instead of ~/.local/share/python-for-android. The python-for-android source code is stored in $PROJECT_DIR/.buildozer/android/platform/python-for-android.

python-for-android 0.6 released

We’ve just officially released python-for-android 0.6. The new version can be downloaded via pip, or will be used by buildozer automatically in new installations. This release contains about 130 new commits from 14 different contributors. Thanks to everyone involved!

python-for-android is a packaging tool for turning Python scripts and apps into Android APKs. It was originally created for use with the Kivy graphical framework, but now supports multiple kinds of Python app including Kivy, PySDL2, a webview interface with Flask or other webserver backend, plain Python scripts without a GUI, or other possibilities such as Python builds for use in other applications.

As planned following the release of python-for-android 0.5, the new version includes some relatively major changes and improvements. In particular, python-for-android should now work with all recent versions of the Android SDK and NDK. On the SDK side this means python-for-android now uses gradle if available, although this doesn’t require any changes to the configuration on the user side.

For full instructions and further information, see the python-for-android documentation.

python-for-android 0.5 released

We’ve just officially released python-for-android 0.5. The new version can be downloaded via pip, or will be used by buildozer automatically in new installations. This release contains about 300 commits from almost 40 different contributors. Thanks to everyone involved!

python-for-android is a packaging tool for turning Python scripts and apps into Android APKs. It was originally created for use with the Kivy graphical framework, but now supports multiple kinds of Python app including Kivy, PySDL2, a webview interface with Flask or other webserver backend, plain Python scripts without a GUI, or other possibilities such as Python builds for use in other applications.

This release contains many fixes and improvements to all parts of the toolchain. Now that these have been released as stable, we intend to move quickly to make some larger improvements including supporting gradle builds, and better support for Python 3 in some recipes.

For full instructions and further information, see the python-for-android documentation.

Pyonic interpreter 1.3 released: Adds support for loading and executing files

I’ve just released Pyonic interpreter 1.3. As usual you can download it from Google Play, for Python 2.7 or Python 3.6. The APKs can also be downloaded directly from Github (where the source code is also available).

The new filebrowser interface in Pyonic interpreter.

The main addition in this release is a file browser interface, which gives Pyonic the ability to load and execute files in the interpreter. This is mildly useful on its own, and I’ve had comments that people would like to be able to do it, but it’s also groundwork for full support for file editing support. I hope to add these features in a future version.