ipykernel 7.x does not schedule async background tasks after cell completion

Start by running the minimal reproducible example in the issue with ipykernel 6.3.1 and 7.2.0, then compare async task scheduling after cell completion. Trace the kernel entry point handling cell completion and background asyncio tasks; done means determining whether the behavior is intentional or a regression and documenting or testing the expected behavior.

Description

In ipykernel 7.x, async background tasks created during a cell (e.g., via asyncio.ensure_future) do not continue running once the cell completes. The task resumes only after executing another cell. The same code behaves correctly on ipykernel 6.x.

This is not about widget rendering or Output; the symptom reproduces with a pure side effect (file write).
(Original discussion history for context: ipywidgets issue list shows jupyter-widgets/ipywidgets#4012 where this started, but this report is deliberately scoped to ipykernel behavior.)

Minimal Reproducible Example

import asyncio
import ipywidgets as W
from IPython.display import display
from IPython import get_ipython

q = asyncio.Queue()
button = W.Button(description="Click me")
current_cell_parent = get_ipython().get_parent()

def on_click(_):
    q.put_nowait("click")

button.on_click(on_click)

async def consumer():
    get_ipython().set_parent(current_cell_parent)
    while True:
        await q.get()
        with open("consumer_log.txt", "a") as f:
            f.write("GOT click\n")

asyncio.ensure_future(consumer())

display(button)

Observed behavior

With ipykernel==6.3.1, clicking the button immediately creates/updates consumer_log.txt.
With ipykernel==7.2.0 or ipykernel==7.1.0, clicking the button does not create/update the file. The file is only written after running another cell.

Expected Behavior

Async tasks created with asyncio.ensure_future (or similar mechanisms) should continue running after the originating cell finishes, just as they do under ipykernel 6.x.

Note on ipykernel PR #1451

Just as a note to help with context: this issue appears to be different from ipython/ipykernel#1451.

That pull request focused on how background thread output is routed to notebook cells, and the ipykernel 7.1.0 release notes describe it as a fix related to display/output behavior coming from threads.

In the situation described here, the behavior involves async background tasks (e.g., asyncio.ensure_future) not continuing to run once a cell has finished executing. The minimal example uses a simple file write to show that no widget mechanisms or thread‑based output routing are involved.

Question

Is this change in async scheduling behavior intentional in ipykernel 7.x?
If so:

• What is the recommended mechanism for running long‑lived async background tasks in notebooks?

If not:

• This appears to be a regression relative to ipykernel 6.x.

Context

• Python: 3.11.9
• JupyterLab: 4.5.6
• ipywidgets: 8.1.8
• ipykernel: 7.2.0