Non-blocking operations

Sometimes in a Shiny app, you need to perform a long-running operation, like loading a large dataset, calling a slow web API, or doing an expensive computation. If you do this in reactive code, it can cause two problems:

The current user’s app stops responding. While a session runs its reactive code, it does not process new input values or update its outputs. The user may think that the app has crashed.
Other users’ apps stop responding. If the operation is synchronous, it blocks the Python process. Every other session that the process serves (to a different user or other tab) must wait until the operation finishes. To those users, the app just feels slow for no reason.

Async reactive code solves the second problem. An ExtendedTask solves both. In this article, we’ll review how Shiny runs async reactive code, and how to use ExtendedTask to keep an app responsive during long-running operations.

Using async/await in Shiny

Asynchronous programming is a technique used in many programming languages to increase scalability and responsiveness, usually in programs that do a lot of networking like web servers and clients. Python supports async programming at the language level, using the async/await keywords; in the standard library, with the asyncio module; and in third-party libraries, like FastAPI and aiohttp.

Shiny is built on top of Starlette, which is an async web framework, so it’s possible to use async functions in many parts of your Shiny app. Shiny is also designed around reactive programming concepts, and these concepts affect what async code can do in a reactive context.

Reactive async/await

Shiny supports the use of async and await in your reactive code. You can use async functions in @reactive.effect, @reactive.calc, and render functions, and within those async functions you can use await to wait for the results of other async functions.

When a reactive function waits at an await, Shiny runs other code in the meantime, including the reactive code of other sessions. Thus, a slow await in one user’s session does not slow down the apps of other users.

However, an await does not automatically make the app more responsive for the current user. The app below displays the current time in a reactive output at the top of the page. Click the button and notice that, during the five seconds that the app is waiting for the (artificially slow) sum calculation to complete, the time does not update.

#| '!! shinylive warning !!': |
#|   shinylive does not work in self-contained HTML documents.
#|   Please set `embed-resources: false` in your metadata.
#| standalone: true
#| components: [editor, viewer]
import asyncio
import datetime
from shiny import App, reactive, render, ui

app_ui = ui.page_fluid(
    ui.p("The time is ", ui.output_text("current_time", inline=True)),
    ui.hr(),
    ui.input_numeric("x", "x", value=1),
    ui.input_numeric("y", "y", value=2),
    ui.input_task_button("btn", "Add numbers"),
    ui.output_text("sum"),
)

def server(input, output, session):
    @render.text
    def current_time():
        reactive.invalidate_later(1)
        return datetime.datetime.now().strftime("%H:%M:%S %p")

    @reactive.calc
    @reactive.event(input.btn)
    async def sum_values():
        await asyncio.sleep(5)
        return input.x() + input.y()

    @render.text
    async def sum():
        return str(await sum_values())

app = App(app_ui, server)

Despite being defined as an asynchronous function, the sum logic still blocks the time output. You could replace await asyncio.sleep(5) with its synchronous equivalent, time.sleep(5), and you’d get exactly the same result. The next section explains this (perhaps surprising) behavior.

How Shiny runs reactive code

Shiny manages the reactive code of each session in cycles. A cycle starts when something invalidates the reactive graph of the session, for example a changed input or an invalidate_later() timer. Shiny then runs each invalidated effect and output of that session, and the calcs that they read. The cycle ends when all of them finish, including any asynchronous parts. Shiny then sends all the new output values to the browser together, and starts the next cycle. This has two results:

New inputs and timers wait for the next cycle. If the user changes an input while a cycle runs, Shiny processes the change after the current cycle ends. The same is true for any invalidate_later() timers. In the example above, the timer of the clock expires while the sum runs, so the clock updates only after the sum finishes and a new cycle begins.
Async code in the same cycle runs concurrently. If two async effects in the same session both call await asyncio.sleep(1), the second one starts while the first one waits, and both finish after about one second. That means there is no guarantee that one async effect finishes before another one starts. If two effects change the same mutable object, each one can see the changes of the other at each await.

Independent sessions of a Shiny app (e.g. in a different tab or for a different user) have their own cycles, so a long cycle in one session does not delay the cycles of other sessions.

Shiny holds inputs (and timers) until the current cycle ends so that all the reactive code in a cycle sees the same values. If Shiny processed a new input in the middle of a cycle, functions that read the input before the change and functions that read it after would get different values. The outputs of that cycle would then disagree with each other, and there would be no way to know which is the “right” value. Because each cycle sees one consistent set of inputs, you can reason about how your app will behave.

Code outside of the reactive graph, like a background task or a custom timer, can change reactive values too. To keep the same consistency, use session.run_once_when_idle(). It runs a function after the session’s current cycle ends, for example session.run_once_when_idle(lambda: latest.set(new_value)).

When to use async reactive code

Using async functions in your reactive code is useful in some common cases:

You need to call a function that is only available as an async function. This includes some functions in Shiny itself that are used to communicate with the browser, like Session.send_custom_message.
The slow part of the work is I/O that an async library can do, like a web request with aiohttp or a database query with an async driver like asyncpg or SQLAlchemy’s async engine. While the request or query waits, other sessions continue to run without pausing.

Async code lets other code run only while it waits at an await. CPU-bound work, and synchronous calls to libraries that are not async, block the Python process even inside an async function.

To keep the current user’s app responsive, use an ExtendedTask.

Responsive apps with `ExtendedTask`

To keep a session responsive while a long-running operation runs, we can move the operation out of the session’s reactive cycle with an ExtendedTask. This works as follows:

Read whatever reactive inputs and calcs will be needed to perform the task.
Perform the task asynchronously and concurrently, outside of the reactive graph.
When the task completes, the resulting value (or error, if the operation failed) becomes available to the reactive graph.

The ExtendedTask class makes this pattern easy to implement. To create an ExtendedTask, use the @reactive.extended_task decorator on an async function. Let’s adapt the example above to use an ExtendedTask:

#| '!! shinylive warning !!': |
#|   shinylive does not work in self-contained HTML documents.
#|   Please set `embed-resources: false` in your metadata.
#| standalone: true
#| components: [editor, viewer]
import asyncio
import datetime
from shiny import App, reactive, render, ui

app_ui = ui.page_fluid(
    ui.p("The time is ", ui.output_text("current_time", inline=True)),
    ui.hr(),
    ui.input_numeric("x", "x", value=1),
    ui.input_numeric("y", "y", value=2),
    ui.input_task_button("btn", "Add numbers"),
    ui.output_text("sum"),
)

def server(input, output, session):
    @render.text
    def current_time():
        reactive.invalidate_later(1)
        return datetime.datetime.now().strftime("%H:%M:%S %p")

    @ui.bind_task_button(button_id="btn")
    @reactive.extended_task
    async def sum_values(x, y):
        await asyncio.sleep(5)
        return x + y

    @reactive.effect
    @reactive.event(input.btn)
    def btn_click():
        sum_values(input.x(), input.y())

    @render.text
    def sum():
        return str(sum_values.result())

app = App(app_ui, server)

Note the sum_values function, which is where the actual (slow) work is done. It is decorated with @reactive.extended_task, which means that it will be run asynchronously and concurrently with other tasks, and that its result will be available as sum_values.result(). Because the task runs outside of the session’s reactive cycle, everything else in the session stays responsive while the task runs. Click the button and notice that the time continues to update during the five seconds of the sum.

We’ve also added a @ui.bind_task_button decorator on top of the @reactive.extended_task decorator.

Note

This synchronizes the ExtendedTask object with the ui.input_task_button in the UI, so that the button will be in its “Processing…” state while the task is running. It does not cause button clicks to invoke the task; we still need to do that manually, which we’ll talk about next.

If you use some other UI gesture or condition besides ui.input_task_button to invoke the task, don’t include the @ui.bind_task_button decorator–it doesn’t work with ui.input_action_button, for example.

See the Task Button component page for the button’s own options, including label_busy and resetting it manually with ui.update_task_button().

Invoking the task

Unlike a reactive effect, simply creating an extended task does not cause it to run. It needs to be invoked (called like a function).

In this case, the sum_values extended task is called from the btn_click reactive effect (@reactive.effect), which runs whenever the button is clicked (@reactive.event(input.btn)).

Notice also that the sum_values logic no longer reads input.x() and input.y() directly in the function body. Because it is now an extended task, attempting to do so would result in an error. Instead, it takes x and y as arguments, which are passed in by btn_click based on reactive inputs.

Retrieving results

The sum output is now a regular synchronous render function, which reads sum_values.result() to get the result of the extended task. This output does not exactly “wait for” the task to complete. Instead, it runs each time the task changes state. For each state, sum_values.result() will behave differently:

Not yet invoked: Raises a silent exception, which will cause the sum output to display nothing.
Running: Raises a special type of exception that tells Shiny to keep the output in the “in progress” state.
Successfully completed: Returns the return value of sum_values, in this case an integer.
Completed with an exception: If sum_values raised an exception while processing, then re-raises that same exception, causing it to be displayed to the user in the sum output.

It’s not necessary to memorize these states. Just remember that sum_values.result() is a reactive, synchronous method that knows how to do the right thing based on the state of the extended task.

Other features of extended tasks

Cancel a running task

Although it’s not shown in the example above, you can also cancel an extended task by calling the cancel() method (for example, sum_values.cancel()). This will attempt to cancel the asyncio task that is running the extended task, and will cause sum_values.result() to raise a silent exception.

Calling sum_values.cancel() on a task that isn’t running will have no effect.

Multiple invocations

An extended task can run concurrently with reactive code and with other extended tasks–that’s its whole purpose. However, a single extended task object cannot run multiple invocations concurrently. If you call sum_values() while it is already running, it will enqueue the new invocation and run it after the first one completes.

This is often not the behavior you want, especially if the task takes a long time to complete. A user may accidentally click an action button twice, or they may click it again because they think the first click didn’t work. To prevent this, use ui.input_task_button instead of ui.input_action_button to invoke the task, since the former automatically prevents subsequent clicks until the task completes.

Executing on a different thread/process

Extended task objects run their tasks using asyncio.create_task, which means that they will run on the same thread as the rest of the application. This is fine for logic that is truly asynchronous, like making a network request via an asyncio-aware library like aiohttp, but it’s not ideal for CPU-bound tasks or when performing I/O synchronously. CPU-bound or synchronous tasks block the main thread, so we’re back to where we started: every session in the Python process must wait until the task completes.

Fortunately, we can rely on Python’s concurrent.futures executors to run code on different threads or processes.

Note

The examples below work well in Shiny Core, but copying them directly into Shiny Express will not work as well. The ThreadPoolExecutor and ProcessPoolExecutor objects need to be created as module-level variables, not as session-level variables, since we’d ideally like to pool resources across all sessions.

Note

ProcessPoolExecutor is not available in Shinylive (e.g. WASM mode). ThreadPoolExecutor is available in Shinylive and appears to work, but doesn’t: it actually performs all of its work on the main thread.

The following example shows how to run a task on a different thread. This is a good strategy for code that does synchronous I/O, like reading from disk or querying a database or remote API. It is a poor fit for CPU-bound code, because Python’s global interpreter lock will prevent the task from running concurrently with other Python code.

import asyncio
import concurrent.futures
import time

from shiny import App, reactive, render, ui

app_ui = ui.page_fluid(
    ui.input_numeric("x", "x", value=1),
    ui.input_numeric("y", "y", value=2),
    ui.input_task_button("btn", "Add numbers"),
    ui.output_text("sum"),
)

# Execute the extended task logic on a different thread. To use a different
# process instead, use concurrent.futures.ProcessPoolExecutor.
pool = concurrent.futures.ThreadPoolExecutor()

def slow_sum(x, y):
    time.sleep(5)  # Simulate a slow synchronous task
    return x + y

def server(input, output, session):
    @ui.bind_task_button(button_id="btn")
    @reactive.extended_task
    async def sum_values(x, y):
        loop = asyncio.get_event_loop()
        return await loop.run_in_executor(pool, slow_sum, x, y)

    @reactive.effect
    @reactive.event(input.btn)
    def btn_click():
        sum_values(input.x(), input.y())

    @render.text
    def sum():
        return str(sum_values.result())

app = App(app_ui, server)
app.on_shutdown(pool.shutdown)

With a small tweak, we can run the task on a different process instead of a different thread: just replace concurrent.futures.ThreadPoolExecutor with concurrent.futures.ProcessPoolExecutor. This is a good strategy for CPU-bound code, because it allows the task to run concurrently with other Python code.

In this example, the slow_sum function is defined at the module level, outside of the Shiny server function. This is critically important for ProcessPoolExecutor to work correctly because of how Python’s multiprocessing module works: only module-level functions can be pickled, which is how multiprocessing sends work to a worker subprocess. (It’s less critical for ThreadPoolExecutor, but still a good programming practice to define such logic at the module level when possible.)

There’s also the app.on_shutdown(pool.shutdown) line at the end of each example. This is necessary to ensure that the pool is shut down when the app stops running. In particular, if you’re using ProcessPoolExecutor and neglect to shut down the pool, you can end up with orphaned Python processes hanging around.

Summary

Key things to remember:

Slow synchronous reactive code blocks every session in the Shiny app process while it runs, so avoid long-running synchronous work in reactive code to keep your app responsive across users.
Async reactive code lets other sessions run while it waits at an await. Use it to call async-only functions, and for slow I/O that an async library can do. Multiple async reactive functions in a session also run concurrently, so their slow operations, like web requests and database queries, overlap instead of running one after another. This does not necessarily make the app more responsive for the current user, because a session processes new inputs and sends new outputs only after all of its reactive code finishes.
To keep the current user’s app responsive, factor your slow code into an ExtendedTask callable object, call it from a reactive effect, and read its result() from a render function or reactive calc. If the slow code is CPU-bound or synchronous, run it on a different thread or process from inside the ExtendedTask.

Monitoring Performance with OpenTelemetry

ExtendedTask operations are automatically traced when using OpenTelemetry. This lets you monitor background task execution, identify slow operations, and track performance in production without adding debug code.

Using async/await in Shiny

Reactive async/await

How Shiny runs reactive code

When to use async reactive code

Responsive apps with ExtendedTask

Invoking the task

Retrieving results

Other features of extended tasks

Cancel a running task

Multiple invocations

Executing on a different thread/process

Summary

Responsive apps with `ExtendedTask`