An actual history of Python on Android

This ZDNet article was published a few days ago about how "Python apps might soon be running on Android". It summarises some recent developments in Android support for CPython, but disappointingly it’s highly misleading about some key points. In particular the article states that "apps written in Python may one day run natively on iOS and Android devices", but in fact people have been doing this since at least 2011.

I thought I’d write some short details about the actual history of Python on Android from my own perspective. It’s something I’ve been involved in on some level for about 8 years, and for a significant part of that I’ve been a primary maintainer of python-for-android. This is a build tool for creating APKs from Python applications, originally created for Kivy but now more generic so that it can also support e.g. flask running on the device with a webview gui, or more recently Pygame via its upcoming SDL2 support.

Kivy app screenshots

Example Python apps for Android. Large version. From left to right: Flat Jewels, ColourBlind, Kognitivo, Lazy Baduk, Pyonic interpreter, Barly (original Python version).

Notes on the term "native"

A key problem point around discussing Python on Android in that the term "native" is overloaded to mean two different things.

The first refers to simply compiling CPython to run "natively" with the Android kernel and libraries, just as it does on desktop platforms, and this is what the ZDNet article refers to with e.g. the quote "Android devices are now fast enough, and the Android kernel permissive enough, to run CPython itself". This is not a recent development, it has been technically possible for many years, although the article is correct that it comes with disadvantages and not everything works the same as on desktop.

The second is using the "native" GUI toolkit for the platform. On Android that means using the same widget toolkit as normal Java-based development. This is the part that is difficult and not yet widely supported, see below for more discussion about the recent developments. I’m actually not sure if the ZDNet article intends to focus on this, but perhaps fails because the author doesn’t understand the distinction, certainly the article focuses on CPython technical details.

It’s absolutely true that having Python applications seamlessly using the native GUI toolkit is essential to making Python a serious alternative to Java for generic Android development. However, there are huge areas of Python development where this simply isn’t a big concern, e.g. everything from games (Pygame, Kivy, Renpy), to scientific visualisation (matplotlib, Vispy, other toolkits), to novel user interfaces (Jupyter notebooks or other apps that don’t care about the native GUI toolkit for whatever reason). All of these things are tremendously popular on the desktop without using the native GUI toolkit, because that detail is either irrelevant or specifically at odds with what they want to do.

History of Python for Android projects

The following projects are ones I remember as interesting and historically important in terms of creating Python applications on Android, in rough date order. This list undoubtedly isn’t complete and is strongly based on events as I remember them, I might have missed other important projects and I’m certainly missing details from before about 2012! I’ve also focused on application build tools, not the many individual patches and Python contributions that made them possible.

Pygame Subset for Android (PGS4A)

Pygame gained rudimentary Android support fairly early in Android’s history. I’m not sure exactly what happened when, but this old release announcement is from February 2011 and isn’t the first version.

PGS4A worked by compiling CPython for Android, with a modified version of SDL (the gui library that Pygame uses), and some Java code to get the app to start and display a surface that pygame could draw to. I believe there were some limitations to what you could do (there’s a reason it was a "subset" of Pygame), but I’m not sure what these were.

I remember people were actively using PGS4A around 2012 when I started looking at it myself, but its popularity slowly diminished over time, probably due to lack of developer support combined with its limitations.

Renpy

Ren’Py is a visual novel engine with great cross-platform support. Actually, I suspect it’s quietly one of the most successful Python game engine projects, e.g. you can find quite a few Renpy-made games available via Steam.

Renpy gained Android support a long time ago, maybe in 2011 or earlier. I believe that as with PGS4A, it worked by combining pygame with a modified version of SDL and some custom Java code to display an app surface that Renpy could draw to. I’m not sure if the Renpy build tools actually had history in common with PGS4A, but I don’t think they were the same project at any point I remember.

Renpy’s Android support hasn’t been static, it’s evolved since 2011. At some point around 2014-2015 it switched to use a fork of Pygame using SDL2, with corresponding updates to the Android build process. This makes sense because SDL2 itself supports Android properly, removing a huge maintenance burden. Note that this pygame_sdl2 project is not the same as the SDL2 support currently under development from the core Pygame team, I don’t think Renpy’s pygame_sdl2 is in wide use and I don’t know what its limitations are.

Kivy and python-for-android

Kivy is easily the most well known Android-supporting Python toolkit I’m aware of, and has been since around 2012 when I came across it myself after failing to get a Java Android tutorial working. Kivy is a graphical toolkit that was not specifically designed for mobile support, but instead focused on being generically cross-platform and supporting novel user interfaces. This turned out very timely, as these properties made it very well suited for Android and iOS support, although Kivy also supports desktop use. Kivy draws its GUI using OpenGL, which has the advantage of working essentially the same on all different platforms, but this can also be a disadvantage in that it means not using the native GUI toolkit as discussed above.

My understanding is that Kivy’s Android support was originally based on Renpy’s Android build tools, which through some amount of collaboration and changes from different places ended up being the genesis for Kivy’s python-for-android project, first commit November 2011. Renpy’s Android build project then shifted to use a fork of python-for-android at some point, after which the projects have been developed separately.

Python-for-android was originally quite Kivy-specific, but was totally rewritten in 2015 to be a more generic and modular build tool, coinciding with Kivy’s own transition to SDL2 as a default backend (in fact this is where I really got involved). Since then its breadth of support has increased dramatically.

Beeware

BeeWare is a collection of tools and libraries for building Python applications across different platforms, both desktop and mobile. These projects have a particular focus on manipulating the native graphical toolkits of a given platform, e.g. on Android they want to use the same "native" GUI widgets as a normal Java-built application. Its toga toolkit provides a platform-independent GUI abstraction for this, in combination with platform-specific toolkits for each individual target.

I believe the BeeWare developers (or at least the core developer Russell Keith-Magee) did some initial experimentation using CPython on Android somewhere around 2015. The idea there would be to create and manipulate the normal Java-native GUI widgets using Java Native Interface (JNI). This is actually possible, it’s something we also support in Kivy and is occasionally useful to e.g. display a webview. Unfortunately it has some key disadvantages including that Python is still quite slow to start, and in particular that Android used to enforce a fairly low limit on the number of JNI references that could be simultaneously maintained, which makes building a full GUI impractical. My understanding is that a combination of these factors made CPython use impractical for Beeware on Android.

BeeWare instead switched to creating VOC, a Python code to Java bytecode transpiler. This converts the Python input into genuine Java bytecode that can run as a normal app without the above limitations. I haven’t tried this for some time, but I understand it works fine. However, it seems the difficulty of supporting the full breadth of Python libraries has been a barrier (at least, that’s been my impression from watching discussions about it, I think it’s still under active development and working well).

Fortunately Android itself has improved, and in particular the limitation on JNI references is no longer present in recent versions. In 2019 BeeWare switched back to targeting CPython on Android, supported by a PSF grant. It is the outcome of this grant that led to the discussions and article I linked at the top. See the end of this post for a brief summary.

Chaquopy

Chaquopy provides build tools for both including Python code in Java applications, and building apps entirely in Python. I’m not sure about the technical details, but I’ve been consistently aware of it as an active project since about 2017, so it may be useful to anyone interested in this sort of thing. I guess there must be some overlap with what python-for-android does, but Chaquopy’s integration of Python and Java code seems to be more of a focus.

pyqtdeploy

The popular Qt graphical framework supports Android. Python bindings to this framework are quite popular, so it’s not a huge surprise that there’s some level of Python for Android support using Qt for the GUI. As far as I’m aware pyqtdeploy is the primary build tool for this, using the PyQt bindings, but I may not be up to date about it. I’ve never seen this to be very popular, but I don’t know if there’s a reason for this beyond its relative obscurity.

What are actually the recent developments in CPython on Android?

Various contributors have driven improvements in CPython’s Android support over the years. I would give more details but honestly I’ve never found the time to get deeply involved so the historical summary is limited by my own ignorance! This has brought CPython to the point of being fairly easy to compile for Android as of about version 3.6. For instance, python-for-android’s CPython build recipe applies no essential patches to the Python 3 source, just an appropriate set of build arguments.

The key recent development is the BeeWare project’s switch to CPython explained above. They’ve made a specific goal of understanding where CPython’s Android support can be improved, and getting involved to resolve these problems. This means attacking both individual technical issues (e.g. getting Python’s test suit passing correctly), and longer-term structural problems (e.g. the suggestion described in the ZDNet article to create a stripped-back Python kernel for mobile use).

These recent developments are great, and hopefully will lead to huge improvements in the ease of deploying Python applications for Android, especially addressing the missing functionality of using the native Java GUI toolkit. However, let’s not forget the history of CPython on Android, people have been creating applications for both business and pleasure for many years.

An actual history of Python on Android

This ZDNet article was published a few days ago about how “Python apps might soon be running on Android”. It summarises some recent developments in Android support for CPython, but disappointingly it’s highly misleading about some key points. In particular the article states that “apps written in Python may one day run natively on iOS and Android devices”, but in fact people have been doing this since at least 2011.

I thought I’d write some short details about the actual history of Python on Android from my own perspective. It’s something I’ve been involved in on some level for about 8 years, and for a significant part of that I’ve been a primary maintainer of python-for-android. This is a build tool for creating APKs from Python applications, originally created for Kivy but now more generic so that it can also support e.g. flask running on the device with a webview gui, or more recently Pygame via its upcoming SDL2 support.

Kivy app screenshots

Example Python apps for Android. Large version. From left to right: Flat Jewels, ColourBlind, Kognitivo, Lazy Baduk, Pyonic interpreter, Barly (original Python version).

Notes on the term “native”

A key problem point around discussing Python on Android in that the term “native” is overloaded to mean two different things.

The first refers to simply compiling CPython to run “natively” with the Android kernel and libraries, just as it does on desktop platforms, and this is what the ZDNet article refers to with e.g. the quote “Android devices are now fast enough, and the Android kernel permissive enough, to run CPython itself”. This is not a recent development, it has been technically possible for many years, although the article is correct that it comes with disadvantages and not everything works the same as on desktop.

The second is using the “native” GUI toolkit for the platform. On Android that means using the same widget toolkit as normal Java-based development. This is the part that is difficult and not yet widely supported, see below for more discussion about the recent developments. I’m actually not sure if the ZDNet article intends to focus on this, but perhaps fails because the author doesn’t understand the distinction, certainly the article focuses on CPython technical details.

It’s absolutely true that having Python applications seamlessly using the native GUI toolkit is essential to making Python a serious alternative to Java for generic Android development. However, there are huge areas of Python development where this simply isn’t a big concern, e.g. everything from games (Pygame, Kivy, Renpy), to scientific visualisation (matplotlib, Vispy, other toolkits), to novel user interfaces (Jupyter notebooks or other apps that don’t care about the native GUI toolkit for whatever reason). All of these things are tremendously popular on the desktop without using the native GUI toolkit, because that detail is either irrelevant or specifically at odds with what they want to do.

History of Python for Android projects

The following projects are ones I remember as interesting and historically important in terms of creating Python applications on Android, in rough date order. This list undoubtedly isn’t complete and is strongly based on events as I remember them, I might have missed other important projects and I’m certainly missing details from before about 2012! I’ve also focused on application build tools, not the many individual patches and Python contributions that made them possible.

Pygame Subset for Android (PGS4A)

Pygame gained rudimentary Android support fairly early in Android’s history. I’m not sure exactly what happened when, but this old release announcement is from February 2011 and isn’t the first version.

PGS4A worked by compiling CPython for Android, with a modified version of SDL (the gui library that Pygame uses), and some Java code to get the app to start and display a surface that pygame could draw to. I believe there were some limitations to what you could do (there’s a reason it was a “subset” of Pygame), but I’m not sure what these were.

I remember people were actively using PGS4A around 2012 when I started looking at it myself, but its popularity slowly diminished over time, probably due to lack of developer support combined with its limitations.

Renpy

Ren’Py is a visual novel engine with great cross-platform support. Actually, I suspect it’s quietly one of the most successful Python game engine projects, e.g. you can find quite a few Renpy-made games available via Steam.

Renpy gained Android support a long time ago, maybe in 2011 or earlier. I believe that as with PGS4A, it worked by combining pygame with a modified version of SDL and some custom Java code to display an app surface that Renpy could draw to. I’m not sure if the Renpy build tools actually had history in common with PGS4A, but I don’t think they were the same project at any point I remember.

Renpy’s Android support hasn’t been static, it’s evolved since 2011. At some point around 2014-2015 it switched to use a fork of Pygame using SDL2, with corresponding updates to the Android build process. This makes sense because SDL2 itself supports Android properly, removing a huge maintenance burden. Note that this pygame_sdl2 project is not the same as the SDL2 support currently under development from the core Pygame team, I don’t think Renpy’s pygame_sdl2 is in wide use and I don’t know what its limitations are.

Kivy and python-for-android

Kivy is easily the most well known Android-supporting Python toolkit I’m aware of, and has been since around 2012 when I came across it myself after failing to get a Java Android tutorial working. Kivy is a graphical toolkit that was not specifically designed for mobile support, but instead focused on being generically cross-platform and supporting novel user interfaces. This turned out very timely, as these properties made it very well suited for Android and iOS support, although Kivy also supports desktop use. Kivy draws its GUI using OpenGL, which has the advantage of working essentially the same on all different platforms, but this can also be a disadvantage in that it means not using the native GUI toolkit as discussed above.

My understanding is that Kivy’s Android support was originally based on Renpy’s Android build tools, which through some amount of collaboration and changes from different places ended up being the genesis for Kivy’s python-for-android project, first commit November 2011. Renpy’s Android build project then shifted to use a fork of python-for-android at some point, after which the projects have been developed separately.

Python-for-android was originally quite Kivy-specific, but was totally rewritten in 2015 to be a more generic and modular build tool, coinciding with Kivy’s own transition to SDL2 as a default backend (in fact this is where I really got involved). Since then its breadth of support has increased dramatically.

Beeware

BeeWare is a collection of tools and libraries for building Python applications across different platforms, both desktop and mobile. These projects have a particular focus on manipulating the native graphical toolkits of a given platform, e.g. on Android they want to use the same “native” GUI widgets as a normal Java-built application. Its toga toolkit provides a platform-independent GUI abstraction for this, in combination with platform-specific toolkits for each individual target.

I believe the BeeWare developers (or at least the core developer Russell Keith-Magee) did some initial experimentation using CPython on Android somewhere around 2015. The idea there would be to create and manipulate the normal Java-native GUI widgets using Java Native Interface (JNI). This is actually possible, it’s something we also support in Kivy and is occasionally useful to e.g. display a webview. Unfortunately it has some key disadvantages including that Python is still quite slow to start, and in particular that Android used to enforce a fairly low limit on the number of JNI references that could be simultaneously maintained, which makes building a full GUI impractical. My understanding is that a combination of these factors made CPython use impractical for Beeware on Android.

BeeWare instead switched to creating VOC, a Python code to Java bytecode transpiler. This converts the Python input into genuine Java bytecode that can run as a normal app without the above limitations. I haven’t tried this for some time, but I understand it works fine. However, it seems the difficulty of supporting the full breadth of Python libraries has been a barrier (at least, that’s been my impression from watching discussions about it, I think it’s still under active development and working well).

Fortunately Android itself has improved, and in particular the limitation on JNI references is no longer present in recent versions. In 2019 BeeWare switched back to targeting CPython on Android, supported by a PSF grant. It is the outcome of this grant that led to the discussions and article I linked at the top. See the end of this post for a brief summary.

Chaquopy

Chaquopy provides build tools for both including Python code in Java applications, and building apps entirely in Python. I’m not sure about the technical details, but I’ve been consistently aware of it as an active project since about 2017, so it may be useful to anyone interested in this sort of thing. I guess there must be some overlap with what python-for-android does, but Chaquopy’s integration of Python and Java code seems to be more of a focus.

pyqtdeploy

The popular Qt graphical framework supports Android. Python bindings to this framework are quite popular, so it’s not a huge surprise that there’s some level of Python for Android support using Qt for the GUI. As far as I’m aware pyqtdeploy is the primary build tool for this, using the PyQt bindings, but I may not be up to date about it. I’ve never seen this to be very popular, but I don’t know if there’s a reason for this beyond its relative obscurity.

What are actually the recent developments in CPython on Android?

Various contributors have driven improvements in CPython’s Android support over the years. I would give more details but honestly I’ve never found the time to get deeply involved so the historical summary is limited by my own ignorance! This has brought CPython to the point of being fairly easy to compile for Android as of about version 3.6. For instance, python-for-android’s CPython build recipe applies no essential patches to the Python 3 source, just an appropriate set of build arguments.

The key recent development is the BeeWare project’s switch to CPython explained above. They’ve made a specific goal of understanding where CPython’s Android support can be improved, and getting involved to resolve these problems. This means attacking both individual technical issues (e.g. getting Python’s test suit passing correctly), and longer-term structural problems (e.g. the suggestion described in the ZDNet article to create a stripped-back Python kernel for mobile use).

These recent developments are great, and hopefully will lead to huge improvements in the ease of deploying Python applications for Android, especially addressing the missing functionality of using the native Java GUI toolkit. However, let’s not forget the history of CPython on Android, people have been creating applications for both business and pleasure for many years.

python-for-android 2019.08.09 released: running under Python 2 no longer supported

python-for-android is a packaging tool for Python apps on Android. You can create your own Python distribution including the modules and dependencies you want, and bundle it in an APK along with your own code.


python-for-android 2019.08.09 has just been released! I haven’t been making a blog post for every new version now that we’ve switched to a monthly releases, but this one includes an especially major change: python-for-android no longer runs under Python 2. If this affects you, simply run python-for-android (or buildozer) under Python 3.6 or newer, they should run exactly the same as ever.

Note that you can still target Python 2 on Android, simply add python2 to your requirements as you would normally. However, this won’t last forever, support for Python 2 will probably be totally removed early in 2020.

Other changes in this release include further testing improvements, significantly improved documentation on developing with python-for-android, and some preparations for further big changes in the future. See the release note for full details.

python-for-android 2019.08.09 released: running under Python 2 no longer supported

python-for-android is a packaging tool for Python apps on Android. You can create your own Python distribution including the modules and dependencies you want, and bundle it in an APK along with your own code.


python-for-android 2019.08.09 has just been released! I haven’t been making a blog post for every new version now that we’ve switched to a monthly releases, but this one includes an especially major change: python-for-android no longer runs under Python 2. If this affects you, simply run python-for-android (or buildozer) under Python 3.6 or newer, they should run exactly the same as ever.

Note that you can still target Python 2 on Android, simply add python2 to your requirements as you would normally. However, this won’t last forever, support for Python 2 will probably be totally removed early in 2020.

Other changes in this release include further testing improvements, significantly improved documentation on developing with python-for-android, and some preparations for further big changes in the future. See the release note for full details.

An update on python-for-android: v2019.06.06 released and future plans

python-for-android is a packaging tool for Python apps on Android. You can create your own Python distribution including the modules and dependencies you want, and bundle it in an APK along with your own code.


python-for-android 2019.06.06 has just been released! This release contains 198 commits from 31 different contributors. Many thanks to everyone involved.

Major changes in this release include:

  • Added support for running your setup.py when packaging your app, enabling your code to be installed as a module in the on-device Python environment. This also makes it easy to build Cython or other compiled components.
  • Added some tooling for requesting and checking for app permissions at runtime.
  • Added initial support for matplotlib.
  • Updated many recipes, and especially the SDL2 backend components, for improved performance and stability on Android.
  • Ongoing improvements to our test coverage and infrastructure.
  • Removed a significant amount of dead code relating to long-deprecated build configurations, and unified other parts of the build to reduce duplication of effort between bootstraps.
  • Updated the release model to target regular, smaller releases.

Of course there have also been a wide range of bugfixes and improvements. See the release notes for a full changelog.

Release model

In this release we’ve transitioned to a calendar-based version system. Future releases will continue to be of the form YYYY.MM.DD. We’re initially targeting one release every four weeks. This scheme represents how python-for-android is normally best used: many changes are driven by updates in the Android ecosystem and build toolchain, and in practice it’s usually best to be working from the most recent possible python-for-android release.

This has been made possible by the hard work of various contributors, setting up and continuing to improve python-for-android’s suite of tests and continuous integration services. In the past we haven’t done a great job of keeping up releases alongside major improvements, but this should now be much more straightforward.

If you’re using buildozer then this doesn’t directly affect you, as buildozer was already configured to use a more recent python-for-android version. Buildozer will now automatically transition to use the latest release, represented by the master branch in the python-for-android git repository.

Future work

We’ve had some questions about Google’s plan to require 64-bit support for apps on Google Play, starting in August 2019. In fact python-for-android already supports this, just pass the required architecture as an argument to the build:

p4a apk --arch=arm64-v8a

That said, we’re working to improve our testing and documentation around 64-bit builds, to make sure everything works as expected.

We also don’t currently support multi-architecture builds in a single output APK. This should be technically possible, but hasn’t ever been a focus because it would significantly increase the APK size. We may revisit this, but in the meantime you can upload one APK of each type to Google Play and python-for-android will automatically handle versioning different architectures correctly (i.e. arm64-v8a is an ‘upgrade’ for devices that support it, so that APK will be preferred).

The other current focus is on improving our test infrastructure, especially increasing test coverage and automation. This should further increase the ease of making regular releases, and our confidence that everything continues to work correctly!

These points are just general goals for python-for-android. There is always other maintenance work to do, and contributions of all types are always welcome.

An update on python-for-android: v2019.06.06 released and future plans

python-for-android is a packaging tool for Python apps on Android. You can create your own Python distribution including the modules and dependencies you want, and bundle it in an APK along with your own code.


python-for-android 2019.06.06 has just been released! This release contains 198 commits from 31 different contributors. Many thanks to everyone involved.

Major changes in this release include:

  • Added support for running your setup.py when packaging your app, enabling your code to be installed as a module in the on-device Python environment. This also makes it easy to build Cython or other compiled components.
  • Added some tooling for requesting and checking for app permissions at runtime.
  • Added initial support for matplotlib.
  • Updated many recipes, and especially the SDL2 backend components, for improved performance and stability on Android.
  • Ongoing improvements to our test coverage and infrastructure.
  • Removed a significant amount of dead code relating to long-deprecated build configurations, and unified other parts of the build to reduce duplication of effort between bootstraps.
  • Updated the release model to target regular, smaller releases.

Of course there have also been a wide range of bugfixes and improvements. See the release notes for a full changelog.

Release model

In this release we’ve transitioned to a calendar-based version system. Future releases will continue to be of the form YYYY.MM.DD. We’re initially targeting one release every four weeks. This scheme represents how python-for-android is normally best used: many changes are driven by updates in the Android ecosystem and build toolchain, and in practice it’s usually best to be working from the most recent possible python-for-android release.

This has been made possible by the hard work of various contributors, setting up and continuing to improve python-for-android’s suite of tests and continuous integration services. In the past we haven’t done a great job of keeping up releases alongside major improvements, but this should now be much more straightforward.

If you’re using buildozer then this doesn’t directly affect you, as buildozer was already configured to use a more recent python-for-android version. Buildozer will now automatically transition to use the latest release, represented by the master branch in the python-for-android git repository.

Future work

We’ve had some questions about Google’s plan to require 64-bit support for apps on Google Play, starting in August 2019. In fact python-for-android already supports this, just pass the required architecture as an argument to the build:

p4a apk --arch=arm64-v8a

That said, we’re working to improve our testing and documentation around 64-bit builds, to make sure everything works as expected.

We also don’t currently support multi-architecture builds in a single output APK. This should be technically possible, but hasn’t ever been a focus because it would significantly increase the APK size. We may revisit this, but in the meantime you can upload one APK of each type to Google Play and python-for-android will automatically handle versioning different architectures correctly (i.e. arm64-v8a is an ‘upgrade’ for devices that support it, so that APK will be preferred).

The other current focus is on improving our test infrastructure, especially increasing test coverage and automation. This should further increase the ease of making regular releases, and our confidence that everything continues to work correctly!

These points are just general goals for python-for-android. There is always other maintenance work to do, and contributions of all types are always welcome.

Running buildozer in a virtual machine

This guide describes how to turn your Kivy/Python app into an APK, by running the buildozer build tool in a virtual machine. This is not the only way to run buildozer, it can work natively on Linux or MacOS or be run from the Windows Subsystem for Linux. See the Kivy documentation for more general instructions.

Creating a Virtual Machine

I’ll be using VirtualBox. Other virtualisation software should also work, but you’ll need to adapt the specific instructions.

We also need a target OS. I recommend Lubuntu 18.04, available here (or direct download link). Lubuntu is a light weight Ubuntu variant. You can also use a different distro if you like, but may need to adapt the later instructions.

Once you have downloaded the Lubuntu iso file, start VirtualBox and press New to create a new virtual machine. You’ll see a dialog like the following:

New VM dialog

Fill in the other options as shown in the image. It’s fine to set a larger memory size if you have enough available, or less may also work fine. Then press Create to continue.

Disk image creation dialog

You now need to select a file size for your virtual hard disk. 15 GB should be sufficient, but it’s safest to double that. Leave the other options unchanged and choose Create to continue.

Now, select your new VM and click Start in the main VirtualBox GUI. You should be prompted to select a virtual optical disk to boot from:

Choose iso to boot

Navigate to your Lubuntu 18.04 iso downloaded earlier, as shown, and press Start to continue. The first screen you see should look something like the following:

Select language

Select your language to see the boot menu:

Choose boot option

Choose the second option, "Install Lubuntu". It doesn’t matter if you accidentally press enter to "Try Lubuntu without installing", in this case there should be an Install Lubuntu application on the desktop that you can click to continue the install process.

You’ll be shown a series of dialogs to help prepare the install process. Clicking through with the defaults is fine, or select other options if you prefer.

The fourth screen will ask what kind of install to use, as shown:

Choose install type

The options shown above should be the defaults, and are what you want to use, so go ahead and continue.

Choose partitioning options

Next, select "Erase disk and install Lubuntu". Note that this is only erasing the (emtpy) virtual disk image created earlier, it won’t affect your host operating system.

Click through again, and you’ll eventually reach the user creation screen. It doesn’t matter what your username is, I used kivyuser:

User creation dialog

Click "Continue" to finally start the install. You’ll be asked a few more questions, but eventually will just have to wait for the installation to complete. This shouldn’t take too long. You’ll be prompted to "Restart Now", which you should go ahead and do.

Restart Now screen

If you have any issues with the virtual machine failing to reboot, go ahead and select Machine > Reset in the VirtualBox menu, it doesn’t matter how you do it as long as the machine is reset. If all goes well, Lubuntu should now automatically boot to a login screen - congratulations, your virtual machine is ready to use! Enter your username and password, and proceed to the next section of instructions.

Login screen

Setting up buildozer

We can now go ahead and set up buildozer ready to build your app. Open an LXTerminal as below:

LXTerminal location in menu

We now have to run a few commands to install everything buildozer needs to run. Run the following command to do so, and enter your user’s password if prompted:

sudo apt-get install python3-pip openjdk-8-jdk autoconf libtool python3-venv

That should give us everything we need for a basic app, so we can go ahead and install buildozer:

.. code-block:: sh
python3 -m venv buildozer-env source buildozer-env/bin/activate pip install buildozer cython

Note that we installed cython as well, this is also required for building the APK.

You only have to create the virtual environment once, but if you reboot the virtual machine you’ll need to run source buildozer-env/bin/activate again. See the Python documentation for more details.

The final step before running buildozer is to have your app ready in the virtual machine. You can access a folder in your host machine using VirtualBox shared folders (in the Devices > Shared Folders menu), but I won’t cover the details here. Note though that if you do this you must copy the folder contents to a different folder within the virtual machine, the buidozer process will not work if run within a shared folder.

In the following instructions I’ll assume you’ve created a folder named app_dir and placed a main.py file inside it containing your application code. Navigate to this folder in the terminal (cd app_dir) and run:

.. code-block:: sh
buildozer init

This will create a buildozer.spec file alongside your main.py:

Creating buildozer.spec

Edit the buildozer.spec to set any options you like. In this example I’ve changed only the title and pacakge.name options:

Editing buildozer.spec

I recommend changing very little for this first build, to make sure everything works. It won’t cause any problems if you edit the buildozer.spec again later.

Running buildozer

We’re now ready to actually build the app into an APK file. Start the process with:

.. code-block:: sh
buildozer -v android debug

The -v option asks for verbose output. This is recommended so that you can keep an eye on what’s happening - the details aren’t too important, but you should be able to see that the process never stops in one place for too long.

Buildozer will now download the Android tools it needs. This may take a while.

At some point you’ll be asked to accept the Android SDK license agreement, which is printed for you as in the following image:

SDK license agreement

At this point, press "y" and then enter to accept the agreement (or abort the process if you don’t agree). This is necessary even if you don’t see any text asking you to do so, due to a bug in buildozer (fixed in the next release).

After downloading everything it needs, buildozer will work through the build process compiling and packaging each of the components for your app. This may take a while, but as long as it doesn’t crash then everything is fine. Future builds will be much faster unless you change the build options, as only the contents of your app itself will need re-packaging.

Eventually the build will complete, you’ll see a screen like the following:

Build complete

That’s it, you’re done! You can find the finished APK in the bin directory, as noted in the final message buildozer prints.

Running buildozer in a virtual machine

This guide describes how to turn your Kivy/Python app into an APK, by running the buildozer build tool in a virtual machine. This is not the only way to run buildozer, it can work natively on Linux or MacOS or be run from the Windows Subsystem for Linux. See the Kivy documentation for more general instructions.

Creating a Virtual Machine

I’ll be using VirtualBox. Other virtualisation software should also work, but you’ll need to adapt the specific instructions.

We also need a target OS. I recommend Lubuntu 18.04, available here (or direct download link). Lubuntu is a light weight Ubuntu variant. You can also use a different distro if you like, but may need to adapt the later instructions.

Once you have downloaded the Lubuntu iso file, start VirtualBox and press New to create a new virtual machine. You’ll see a dialog like the following:

New VM dialog

Fill in the other options as shown in the image. It’s fine to set a larger memory size if you have enough available, or less may also work fine. Then press Create to continue.

Disk image creation dialog

You now need to select a file size for your virtual hard disk. 15 GB should be sufficient, but it’s safest to double that. Leave the other options unchanged and choose Create to continue.

Now, select your new VM and click Start in the main VirtualBox GUI. You should be prompted to select a virtual optical disk to boot from:

Choose iso to boot

Navigate to your Lubuntu 18.04 iso downloaded earlier, as shown, and press Start to continue. The first screen you see should look something like the following:

Select language

Select your language to see the boot menu:

Choose boot option

Choose the second option, “Install Lubuntu”. It doesn’t matter if you accidentally press enter to “Try Lubuntu without installing”, in this case there should be an Install Lubuntu application on the desktop that you can click to continue the install process.

You’ll be shown a series of dialogs to help prepare the install process. Clicking through with the defaults is fine, or select other options if you prefer.

The fourth screen will ask what kind of install to use, as shown:

Choose install type

The options shown above should be the defaults, and are what you want to use, so go ahead and continue.

Choose partitioning options

Next, select “Erase disk and install Lubuntu”. Note that this is only erasing the (emtpy) virtual disk image created earlier, it won’t affect your host operating system.

Click through again, and you’ll eventually reach the user creation screen. It doesn’t matter what your username is, I used kivyuser:

User creation dialog

Click “Continue” to finally start the install. You’ll be asked a few more questions, but eventually will just have to wait for the installation to complete. This shouldn’t take too long. You’ll be prompted to “Restart Now”, which you should go ahead and do.

Restart Now screen

If you have any issues with the virtual machine failing to reboot, go ahead and select Machine > Reset in the VirtualBox menu, it doesn’t matter how you do it as long as the machine is reset. If all goes well, Lubuntu should now automatically boot to a login screen - congratulations, your virtual machine is ready to use! Enter your username and password, and proceed to the next section of instructions.

Login screen

Setting up buildozer

We can now go ahead and set up buildozer ready to build your app. Open an LXTerminal as below:

LXTerminal location in menu

We now have to run a few commands to install everything buildozer needs to run. Run the following command to do so, and enter your user’s password if prompted:

sudo apt-get install python3-pip openjdk-8-jdk autoconf libtool python3-venv

That should give us everything we need for a basic app, so we can go ahead and install buildozer:

python3 -m venv buildozer-env
source buildozer-env/bin/activate
pip install buildozer cython

Note that we installed cython as well, this is also required for building the APK.

You only have to create the virtual environment once, but if you reboot the virtual machine you’ll need to run source buildozer-env/bin/activate again. See the Python documentation for more details.

The final step before running buildozer is to have your app ready in the virtual machine. You can access a folder in your host machine using VirtualBox shared folders (in the Devices > Shared Folders menu), but I won’t cover the details here. Note though that if you do this you must copy the folder contents to a different folder within the virtual machine, the buidozer process will not work if run within a shared folder.

In the following instructions I’ll assume you’ve created a folder named app_dir and placed a main.py file inside it containing your application code. Navigate to this folder in the terminal (cd app_dir) and run:

buildozer init

This will create a buildozer.spec file alongside your main.py:

Creating buildozer.spec

Edit the buildozer.spec to set any options you like. In this example I’ve changed only the title and pacakge.name options:

Editing buildozer.spec

I recommend changing very little for this first build, to make sure everything works. It won’t cause any problems if you edit the buildozer.spec again later.

Running buildozer

We’re now ready to actually build the app into an APK file. Start the process with:

buildozer -v android debug

The -v option asks for verbose output. This is recommended so that you can keep an eye on what’s happening - the details aren’t too important, but you should be able to see that the process never stops in one place for too long.

Buildozer will now download the Android tools it needs. This may take a while.

At some point you’ll be asked to accept the Android SDK license agreement, which is printed for you as in the following image:

SDK license agreement

At this point, press “y” and then enter to accept the agreement (or abort the process if you don’t agree). This is necessary even if you don’t see any text asking you to do so, due to a bug in buildozer (fixed in the next release).

After downloading everything it needs, buildozer will work through the build process compiling and packaging each of the components for your app. This may take a while, but as long as it doesn’t crash then everything is fine. Future builds will be much faster unless you change the build options, as only the contents of your app itself will need re-packaging.

Eventually the build will complete, you’ll see a screen like the following:

Build complete

That’s it, you’re done! You can find the finished APK in the bin directory, as noted in the final message buildozer prints.

Kivy Hackathon and FOSDEM 2019

For the last 3 days several of the Kivy Core Developers gathered in Brussels, Belgium for the first ever core developer hackathon. Not only is this the first time we’ve gathered to work on Kivy framework issues, but for most of us the first time we’ve ever met - even after years of collaboration!

From left to right: Peter Badida (KeyWeeUsr), Gabriel Pettier (tshirtman), Mathieu Virbel (tito), Alexander Taylor (inclement) and Andre Miras (AndreMiras)

From left to right the attendees are Peter Badida (KeyWeeUsr), Gabriel Pettier (tshirtman), Mathieu Virbel (tito), Alexander Taylor (inclement) and Andre Miras (AndreMiras).

We used this opportunity to make amazing progress on a range of Kivy issues. Most especially:

  • 55 closed issues and 36 merged pull requests across all Kivy projects, specifically targeting long standing bugs and the most important current issues.
  • A full update of buildozer to work well with the latest SDK, NDK and other dependencies for Android development. New release to follow!
  • Rapid progress on ncis, a new remote inspector/debugger for all types of Python application.
  • Many improvements to the python-for-android continuous integration, and some major bugfixes for stability with Python 3 and the latest toolchain. Preliminary 0.7.0 release already available via PyPI and buildozer, more to follow.
  • Extensive cleanup and improvements for PyJNIus.
  • A further ~170 closed python-for-android issues and ~110 buildozer issues, as a general clean up of stale and outdated reports to focus on the ones that really matter.

For the next two days, the same team will be attending FOSDEM 2019. Let us know by discord or other communication if you’d like to say hello!

Kivy Hackathon and FOSDEM 2019

For the last 3 days several of the Kivy Core Developers gathered in Brussels, Belgium for the first ever core developer hackathon. Not only is this the first time we’ve gathered to work on Kivy framework issues, but for most of us the first time we’ve ever met - even after years of collaboration!

From left to right: Peter Badida (KeyWeeUsr), Gabriel Pettier (tshirtman), Mathieu Virbel (tito), Alexander Taylor (inclement) and Andre Miras (AndreMiras)

From left to right the attendees are Peter Badida (KeyWeeUsr), Gabriel Pettier (tshirtman), Mathieu Virbel (tito), Alexander Taylor (inclement) and Andre Miras (AndreMiras).

We used this opportunity to make amazing progress on a range of Kivy issues. Most especially:

  • 55 closed issues and 36 merged pull requests across all Kivy projects, specifically targeting long standing bugs and the most important current issues.
  • A full update of buildozer to work well with the latest SDK, NDK and other dependencies for Android development. New release to follow!
  • Rapid progress on ncis, a new remote inspector/debugger for all types of Python application.
  • Many improvements to the python-for-android continuous integration, and some major bugfixes for stability with Python 3 and the latest toolchain. Preliminary 0.7.0 release already available via PyPI and buildozer, more to follow.
  • Extensive cleanup and improvements for PyJNIus.
  • A further ~170 closed python-for-android issues and ~110 buildozer issues, as a general clean up of stale and outdated reports to focus on the ones that really matter.

For the next two days, the same team will be attending FOSDEM 2019. Let us know by discord or other communication if you’d like to say hello!