Producer/Consumer model in Kivy

(update: original title was “Publisher/Consumer model with Kivy”, but the literature usually refer to this as “Producer/Consumer”)

Few things are worse to an user than an unresponsive UI, well i can think of a crashing UI, of course, but not much more. So, in an event driven environment, it’s important to avoid blocking the UI for too long.

But sometime you have a task that will take an unacceptable time for such constraint, if the task can’t really be chunked, a Thread is likely to be the acceptable solution, but threads have constraints and in Kivy, you can’t update the UI from one, you have to schedule something to happen on the main thread, and update things from here. If the task is chunkable, it’s even easier, but the following idea can apply to both situation.

So, a solution that i find convenient, is to use a producer/consumer model.

The idea is simple, have a scheduled action each frame do a small part of your task, until a timeout is triggered, and then wait for next frame to continue.
To trigger work, just put (produce) it in a list of tasks to be treated py the consumer.

So let’s start by setting a consumer for adding elements to a list, we don’t want to add 100 elements in the same frame, because that would take too much time.

from kivy.app import App
from kivy.clock import Clock
from kivy.lang import Builder
from kivy.properties import ListProperty
from kivy.uix.label import Label

kv = '''
BoxLayout:
    ScrollView:
        GridLayout:
            cols: 1
            id: target
            size_hint: 1, None
            height: self.minimum_height
     Button:
        text: 'add 100'
        on_press: app.consumables.extend(range(100))
'''

class ProdConApp(App):
    consumables = ListProperty([])

    def build(self):
        Clock.schedule_interval(self.consume, 0)
        return Builder.load_string(kv)'

    def consume(self, *args):
        if self.consumables:
            item = self.consumables.pop(0)
            label = Label(text='%s' % item)
            self.root.ids.target.add_widget(label)

if __name__ == '__main__':
    ProdConApp().run()

Now, i’m only taking one item each frame, it’s probably good enough, but if we have a lot of item, we still may want to take as much is possible, considering kivy loop is frame-limited to 60fps, 100 items will take more than a second, why wait if we have the power? Let’s use a slightly smarter version.

add this import near of the top

from kivy.clock import _default_time as time

then change consume definition to be:

def consume(self, *args):
    limit = Clock.get_time() + 1 / 60.
    while self.consumables and time() < limit:
        item = self.consumables.pop(0)
        label = Label(text='%s' % item)
        self.root.ids.target.add_widget(label)

Of course, this only work because we know creating and adding one widget takes considerably less time than one frame, so it’s not like one of such operation will make our loop hang too long.

Now, filling the consumable list here is done from main UI, but it could totally be done from a Thread, assuming locking is correctly handled (or that extend/pop are atomic, which seems to be the case), so if your filling of work to be displayed is slow, doing the exact same thing as a background task will allow you to do heavy lifting, while keeping your app snappy.

A slightly more demonstrative version of this example can be found here