`run_test()` doesn't install the eager task factory, so tasks start in a different order than under `run()`

Open
#6,792 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
22/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
testing

Research direction

Compare how run_async() installs asyncio.eager_task_factory with how run_test() starts the app in the Textual app module, then reproduce the ordering script from the issue. Done means run_test() matches run() and the three failing tests in tests/command_palette/test_declare_sources.py pass. Those tests depend on the screen-startup behaviour tracked in #6444, so that needs settling first, and a maintainer should confirm the direction before you start.

Written by the indexing model from the issue text.

Description

Hi! I hit this while comparing an app's behaviour under run() with the same app under run_test(). Since #6102, run_async() sets asyncio.eager_task_factory on the loop (Python 3.12+), but run_test() doesn't, so a task created in a handler starts at a different point depending on how the app is run. #6492 mentions this too.

import asyncio
import sys

import textual
from textual.app import App

class OrderApp(App[list]):
    def on_mount(self) -> None:
        self.order = []

        async def task() -> None:
            self.order.append("task started")

        asyncio.create_task(task())
        self.order.append("on_mount continued")
        self.set_timer(0.1, lambda: self.exit(self.order))

async def under_run_test() -> list:
    app = OrderApp()
    async with app.run_test() as pilot:
        await pilot.pause(0.2)
    return app.order

print(f"Textual {textual.__version__}, Python {sys.version.split()[0]}")
print("run():     ", OrderApp().run(headless=True))
print("run_test():", asyncio.run(under_run_test()))

Output on Textual 8.2.7 and on main (5e5b7ef), Python 3.13:

Textual 8.2.7, Python 3.13.18
run():      ['task started', 'on_mount continued']
run_test(): ['on_mount continued', 'task started']

So a test can pass while the real app behaves differently, or the other way round. #6444 is one example: it crashes under run() but not under run_test().

The obvious fix is to set the same factory in run_test() (and restore the previous one on exit). I tried that on main and it's not quite enough on its own:

  • The order above then matches run().
  • Three tests in tests/command_palette/test_declare_sources.py fail (test_no_screen_command_sources, test_screen_command_sources, test_app_and_screen_command_sources_combine). They push a screen in on_mount whose own on_mount opens the command palette. With the eager factory, run_test() gives a screen stack of ['Screen', 'CommandPalette', 'PaletteOnMount'], so the palette ends up under the screen that opened it. Under run() the same app gives ['Screen', 'PaletteOnMount', 'CommandPalette']. I think that's the same problem as #6444 (a screen's children start before the screen is on the stack), so there may be a startup difference between the two paths beyond the task factory.
  • The rest of the suite is unchanged.

Is run_test() meant to match run() here? If so, I'm happy to open a PR that sets the factory, but the three tests above would need #6444 (or a change to how run_test() starts the app) first, so I wanted to check which way you'd prefer.

Dominant language
Python
Stars
37.7k
Forks
1.5k
PR merge metrics
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Textualize/textual

All issues in Textualize/textual

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.