How do you run a blocking method asynchronously with Gio.Task in a Python+GTK app?

Question

I want to have a Python (3) GTK (3) application working as follows:

Figure 1. Views of the application: A, Initial view. B, Progress view. C, Lunch ready view. D, Lunch burnt view.

User clicks on "Make lunch" (view A)
App displays Progress view (view B)
App starts a make_lunch [blocking] operation in the background
Depending on make_lunch result:
1. App displays Lunch ready view (view C) or
2. App displays Lunch burnt view (view D)

What I've tried

I started by writing a synchronous application (see App: synchronous version below). In this version, the GUI gets blocked in the Initial view (view A) when the user clicks the "Make lunch" button. The view is blocked for as long as the make_lunch operation lasts (5 seconds). The Progress view (view B) is never displayed. Instead, it goes directly from view A to any of the result views, C or D, depending on the outcome of the blocking operation. All this, obviously, is what I want to avoid.

If I understand GNOME Developer documentation correctly (see Threads and Asynchronous programming), I need to use Gio.Task (aka GTask) to run the blocking operation in a worker thread.

So, I started translating the standard pattern for asynchronous calls given in the C examples on those documents into Python (see App:Asynchronous version below), but I didn't get far because I'm not familiar with C and because I started getting errors like

ValueError: Pointer arguments are restricted to integers,
capsules, and None. See:
https://bugzilla.gnome.org/show_bug.cgi?id=683599

when calling different methods of the Gio.Task object, like

task.set_source_tag(self.make_lunch_async)
task.set_task_data(task_data)
task.run_in_thread(self.make_lunch_async_callback)

I couldn't get much from the bug report linked from the error messages, so I am stuck.

App: Synchronous version

Here is a blocking version of the application to use as a starting point, an app.py file and an app-window.ui file, which are to be saved in the same directory as follows:

my-blocking-app/
├── app.py
└── app-window.ui

Python code (save as app.py):

import random
import time
import sys
import gi

gi.require_version("Gtk", "3.0")

from gi.repository import Gio, Gtk


# VIEWS

class App(Gtk.Application):
    def __init__(self):
        Gtk.Application.__init__(
            self,
            application_id="org.example.app",
            flags=Gio.ApplicationFlags.FLAGS_NONE
        )

    def do_activate(self):
        # Show default application window.
        window = AppWindow(self)
        self.add_window(window)
        window.show_all()


@Gtk.Template(filename="app-window.ui")
class AppWindow(Gtk.ApplicationWindow):
    __gtype_name__ = "AppWindow"

    start_button = Gtk.Template.Child("start-button")
    operation_box = Gtk.Template.Child("operation-box")
    progress_box = Gtk.Template.Child("progress-box")
    spinner = Gtk.Template.Child("spinner")
    success_box = Gtk.Template.Child("success-box")
    success_label = Gtk.Template.Child("success-label")
    failure_box = Gtk.Template.Child("failure-box")
    failure_label = Gtk.Template.Child("failure-label")
    back_button = Gtk.Template.Child("back-button")

    def __init__(self, app):
        super().__init__()
        self.set_application(app)

    @Gtk.Template.Callback()
    def on_start_button_clicked(self, button):
        """Handle the BUTTON's clicked signal."""
        self.operation_box.set_visible(False)
        self.progress_box.set_visible(True)
        self.spinner.start()

        # FIXME: Blocking operation. Run asynchronously.
        cook = Cook()
        result = cook.make_lunch("rice", "lentils", "carrots")

        # Show result.
        self.show_result(result)

    @Gtk.Template.Callback()
    def on_back_button_clicked(self, button):
        """Handle the BUTTON's clicked signal."""
        self.operation_box.set_visible(True)
        self.success_box.set_visible(False)
        self.failure_box.set_visible(False)
        button.set_visible(False)

    def show_result(self, result):
        """Update application according to result."""
        self.progress_box.set_visible(False)
        self.back_button.set_visible(True)

        if isinstance(result, Plate):
            message = "Lunch is ready: {}".format(result)
            self.success_label.set_text(message)
            self.success_box.set_visible(True)
        else:
            message = result.get("Error")
            self.failure_label.set_text(message)
            self.failure_box.set_visible(True)


# MODELS

class Plate():
    def __init__(self, ingredients):
        self.ingredients = ingredients

    def __str__(self):
        return ", ".join(self.ingredients)


class Cook():
    def make_lunch(self, *ingredients):
        time.sleep(5)

        outcomes = [
            Plate(ingredients),
            {"Error": "Lunch is burned!!"}
        ]

        return random.choice(outcomes)


# RUN APP
if __name__ == "__main__":
    app = App()
    exit_status = app.run(sys.argv)
    sys.exit(exit_status)

XML UI (save as app-window.ui):

Running the app requires the following software in the environment:

gobject-introspection
gtk3
python3
pygobject

App: Asynchronous version (not working)

This adds the methods AppWindow.make_lunch_async, AppWindow.make_lunch_finish and AppWindow.make_lunch_async_callback, trying to mimic the standard pattern for async operation, but couldn't even define a working make_lunch_async because of the errors indicated in the comments. Don't know how to properly translate the C into Python.

import random
import time
import sys
import gi

gi.require_version("Gtk", "3.0")

from gi.repository import Gio, Gtk


# VIEWS

class App(Gtk.Application):
    def __init__(self):
        Gtk.Application.__init__(
            self,
            application_id="org.example.app",
            flags=Gio.ApplicationFlags.FLAGS_NONE
        )

    def do_activate(self):
        # Show default application window.
        window = AppWindow(self)
        self.add_window(window)
        window.show_all()


@Gtk.Template(filename="app-window.ui")
class AppWindow(Gtk.ApplicationWindow):
    __gtype_name__ = "AppWindow"

    start_button = Gtk.Template.Child("start-button")
    operation_box = Gtk.Template.Child("operation-box")
    progress_box = Gtk.Template.Child("progress-box")
    spinner = Gtk.Template.Child("spinner")
    success_box = Gtk.Template.Child("success-box")
    success_label = Gtk.Template.Child("success-label")
    failure_box = Gtk.Template.Child("failure-box")
    failure_label = Gtk.Template.Child("failure-label")
    back_button = Gtk.Template.Child("back-button")

    def __init__(self, app):
        super().__init__()
        self.set_application(app)

    @Gtk.Template.Callback()
    def on_start_button_clicked(self, button):
        """Handle the BUTTON's clicked signal."""
        self.operation_box.set_visible(False)
        self.progress_box.set_visible(True)
        self.spinner.start()

        # Make lunch asynchronously.
        self.make_lunch_async(
            Cook(),
            ("rice", "lentils", "carrots"),
            None,  # Operation is not cancellable.
            self.make_lunch_async_callback,
            None   # No aditional data for the callback
        )

    def make_lunch_async_callback(self, task, source_object, task_data, cancellable):
        """Handle the result of the async operation."""
        cook = task_data.get("cook")
        ingredients = task_data.get("ingredients")
        result = cook.make_lunch(*ingredients)

        # Show result (should I call this here?).
        self.show_result(result)

    def make_lunch_async(self, cook, ingredients, cancellable, callback, callback_data):
        """Schedule async operation and invoke callback when operation
        is done."""
        task = Gio.Task.new(
            self,
            cancellable,
            callback,
            callback_data
        )
        task.set_source_tag(self.make_lunch_async)  # FIXME.
        # Previous line fails with:
        #
        # ValueError: Pointer arguments are restricted to integers,
        # capsules, and None. See:
        # https://bugzilla.gnome.org/show_bug.cgi?id=683599

        # Cancellation should be handled manually using mechanisms
        # specific to the blocking function.
        task.set_return_on_cancel(False)

        # Set up a closure containing the call’s parameters. Copy them
        # to avoid locking issues between the calling thread and the
        # worker thread.
        task_data = {"cook": cook, "ingredients": ingredients}
        task.set_task_data(task_data)  # FIXME.
        # Previous line fails with:
        #
        # ValueError: Pointer arguments are restricted to integers,
        # capsules, and None. See:
        # https://bugzilla.gnome.org/show_bug.cgi?id=683599

        # Run the task in a worker thread and return immediately while
        # that continues in the background. When it’s done it will call
        # @callback in the current thread default main context.
        task.run_in_thread(self.make_lunch_async_callback)

    def make_lunch_finish(self, result, error):
        """What's the purpose of this method."""
        pass

    @Gtk.Template.Callback()
    def on_back_button_clicked(self, button):
        """Handle the BUTTON's clicked signal."""
        self.operation_box.set_visible(True)
        self.success_box.set_visible(False)
        self.failure_box.set_visible(False)
        button.set_visible(False)

    def show_result(self, result):
        """Update application according to result."""
        self.progress_box.set_visible(False)
        self.back_button.set_visible(True)

        if isinstance(result, Plate):
            message = "Lunch is ready: {}".format(result)
            self.success_label.set_text(message)
            self.success_box.set_visible(True)
        else:
            message = result.get("Error")
            self.failure_label.set_text(message)
            self.failure_box.set_visible(True)


# MODELS

class Plate():
    def __init__(self, ingredients):
        self.ingredients = ingredients

    def __str__(self):
        return ", ".join(self.ingredients)


class Cook():
    def make_lunch(self, *ingredients):
        time.sleep(5)

        outcomes = [
            Plate(ingredients),
            {"Error": "Lunch is burned!!"}
        ]

        return random.choice(outcomes)


# RUN APP
if __name__ == "__main__":
    app = App()
    exit_status = app.run(sys.argv)
    sys.exit(exit_status)

user_14369995 · Answer

Using Mazhar Hussain's answer on GNOME Discourse, I updated the example application to run the blocking operation asynchronously without modifying the Cook model. Instead, I defined a reusable AsyncWorker class, whose objects take care of running a given operation in the background.

"""Running an asynchronous operation from GTK App with Gio.Task."""

import random
import time
import sys
import gi

gi.require_version("Gtk", "3.0")

from gi.repository import Gio, GObject, Gtk


# VIEWS

class App(Gtk.Application):
    def __init__(self):
        Gtk.Application.__init__(
            self,
            application_id="org.example.app",
            flags=Gio.ApplicationFlags.FLAGS_NONE
        )

    def do_activate(self):
        # Show default application window.
        window = AppWindow(self)
        self.add_window(window)
        window.show_all()


@Gtk.Template(filename="app-window.ui")
class AppWindow(Gtk.ApplicationWindow):
    __gtype_name__ = "AppWindow"

    start_button = Gtk.Template.Child("start-button")
    operation_box = Gtk.Template.Child("operation-box")
    progress_box = Gtk.Template.Child("progress-box")
    spinner = Gtk.Template.Child("spinner")
    success_box = Gtk.Template.Child("success-box")
    success_label = Gtk.Template.Child("success-label")
    failure_box = Gtk.Template.Child("failure-box")
    failure_label = Gtk.Template.Child("failure-label")
    back_button = Gtk.Template.Child("back-button")

    def __init__(self, app):
        super().__init__()
        self.set_application(app)

    @Gtk.Template.Callback()
    def on_start_button_clicked(self, button):
        """Handle the BUTTON's clicked signal."""
        self.operation_box.set_visible(False)
        self.progress_box.set_visible(True)
        self.spinner.start()

        # Make lunch asynchronously.
        cook = Cook()
        ingredients = ("rice", "lentils", "carrots")
        async_worker = AsyncWorker(
            operation=cook.make_lunch,
            operation_inputs=ingredients,
            operation_callback=self.on_lunch_finished
        )
        async_worker.start()

    @Gtk.Template.Callback()
    def on_back_button_clicked(self, button):
        """Handle the BUTTON's clicked signal."""
        self.operation_box.set_visible(True)
        self.success_box.set_visible(False)
        self.failure_box.set_visible(False)
        button.set_visible(False)

    def on_lunch_finished(self, worker, result, handler_data):
        """Handle the RESULT of the asynchronous operation performed by
        WORKER.

        WORKER (AsyncWorker)
          The worker performing the asynchronous operation.

        RESULT (Gio.AsyncResult)
          The asynchronous result of the asynchronous operation.

        HANDLER_DATA (None)
          Additional data passed to this handler by the worker when the
          job is done. It should be None in this case.

        """
        outcome = worker.return_value(result)
        self.show_outcome(outcome)

    def show_outcome(self, outcome):
        """Update application according to the given outcome."""
        self.spinner.stop()
        self.progress_box.set_visible(False)
        self.back_button.set_visible(True)

        if isinstance(outcome, Plate):
            message = "Lunch is ready: {}".format(outcome)
            self.success_label.set_text(message)
            self.success_box.set_visible(True)
        else:
            message = outcome.get("Error")
            self.failure_label.set_text(message)
            self.failure_box.set_visible(True)


# MODELS

class Plate():
    def __init__(self, ingredients):
        self.ingredients = ingredients

    def __str__(self):
        return ", ".join(self.ingredients)


class Cook():
    def make_lunch(self, *ingredients):
        time.sleep(5)

        outcomes = [
            Plate(ingredients),
            {"Error": "Lunch is burned!!"}
        ]

        return random.choice(outcomes)


# ASYNCHRONOUS WORKER

class AsyncWorker(GObject.Object):
    """Represents an asynchronous worker.

    An async worker's job is to run a blocking operation in the
    background using a Gio.Task to avoid blocking the app's main thread
    and freezing the user interface.

    The terminology used here is closely related to the Gio.Task API.

    There are two ways to specify the operation that should be run in
    the background:

    1. By passing the blocking operation (a function or method) to the
       constructor.
    2. By defining the work() method in a subclass.

    An example of (1) can be found in AppWindow.on_start_button_clicked.

    Constructor parameters:

    OPERATION (callable)
      The function or method that needs to be run asynchronously. This
      is only necessary when using a direct instance of AsyncWorker, not
      when using an instance of a subclass of AsyncWorker, in which case
      an AsyncWorker.work() method must be defined by the subclass
      instead.

    OPERATION_INPUTS (tuple)
      Input data for OPERATION, if any.

    OPERATION_CALLBACK (callable)
      A function or method to call when the OPERATION is complete.

      See AppWindow.on_lunch_finished for an example of such callback.

    OPERATION_CALLBACK_INPUTS (tuple)
      Optional. Additional input data for OPERATION_CALLBACK.

    CANCELLABLE (Gio.Cancellable)
      Optional. It defaults to None, meaning that the blocking
      operation is not cancellable.

    """
    def __init__(
            self,
            operation=None,
            operation_inputs=(),
            operation_callback=None,
            operation_callback_inputs=(),
            cancellable=None
    ):
        super().__init__()
        self.operation = operation
        self.operation_inputs = operation_inputs
        self.operation_callback = operation_callback
        self.operation_callback_inputs = operation_callback_inputs
        self.cancellable = cancellable

        # Holds the actual data referenced from the Gio.Task created
        # in the AsyncWorker.start method.
        self.pool = {}

    def start(self):
        """Schedule the blocking operation to be run asynchronously.

        The blocking operation is either self.operation or self.work,
        depending on how the AsyncWorker was instantiated.

        This method corresponds to the function referred to as
        "blocking_function_async" in GNOME Developer documentation.

        """
        task = Gio.Task.new(
            self,
            self.cancellable,
            self.operation_callback,
            self.operation_callback_inputs
        )

        if self.cancellable is None:
            task.set_return_on_cancel(False)  # The task is not cancellable.

        data_id = id(self.operation_inputs)
        self.pool[data_id] = self.operation_inputs
        task.set_task_data(
            data_id,
            # FIXME: Data destroyer function always gets None as argument.
            #
            # This function is supposed to take as an argument the
            # same value passed as data_id to task.set_task_data, but
            # when the destroyer function is called, it seems it always
            # gets None as an argument instead. That's why the "key"
            # parameter is not being used in the body of the anonymous
            # function.
            lambda key: self.pool.pop(data_id)
        )

        task.run_in_thread(self._thread_callback)

    def _thread_callback(self, task, worker, task_data, cancellable):
        """Run the blocking operation in a worker thread."""
        # FIXME: task_data is always None for Gio.Task.run_in_thread callback.
        #
        # The value passed to this callback as task_data always seems to
        # be None, so we get the data for the blocking operation as
        # follows instead.
        data_id = task.get_task_data()
        data = self.pool.get(data_id)

        # Run the blocking operation.
        if self.operation is None:  # Assume AsyncWorker was extended.
            outcome = self.work(*data)
        else:  # Assume AsyncWorker was instantiated directly.
            outcome = self.operation(*data)

        task.return_value(outcome)

    def return_value(self, result):
        """Return the value of the operation that was run
        asynchronously.

        This method corresponds to the function referred to as
        "blocking_function_finish" in GNOME Developer documentation.

        This method is called from the view where the asynchronous
        operation is started to update the user interface according
        to the resulting value.

        RESULT (Gio.AsyncResult)
          The asyncronous result of the blocking operation that is
          run asynchronously.

        RETURN VALUE (object)
          Any of the return values of the blocking operation. If
          RESULT turns out to be invalid, return an error dictionary
          in the form

          {"AsyncWorkerError": "Gio.Task.is_valid returned False."}

        """
        value = None

        if Gio.Task.is_valid(result, self):
            value = result.propagate_value().value
        else:
            error = "Gio.Task.is_valid returned False."
            value = {"AsyncWorkerError": error}

        return value


# RUN APP
if __name__ == "__main__":
    app = App()
    exit_status = app.run(sys.argv)
    sys.exit(exit_status)

How do you run a blocking method asynchronously with Gio.Task in a Python+GTK app?

What I've tried

App: Synchronous version

App: Asynchronous version (not working)

1 Answers