bcm2835-codec: no V4L2_EVENT_SOURCE_CHANGE after re-initialising the decoder on the same file handle (STREAMOFF on both queues)

Offen
#7,770 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
48/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
c
Bereich
backend

Rechercherichtung

Start with drivers/staging/vc04_services/bcm2835-codec/bcm2835-v4l2-codec.c, especially bcm2835_codec_stop_streaming(), start_streaming(), and queue_setup(), which the proposed diff identifies. Check the decoder stop and restart lifecycle against the V4L2 stateful decoder requirements in Documentation/userspace-api/media/v4l/dev-decoder.rst, and investigate whether component recreation and CAPTURE format reset are safe. Done means a same-handle restart reliably emits SOURCE_CHANGE and decodes frames, including when the new stream has the same resolution.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Device: Raspberry Pi 4 Model B
Kernel: mainline 6.18 LTS with the bcm2835-codec / vc-sm-cma / vchiq-mmal drivers backported from rpi-6.18.y @ ea17ae02ca28 (the codec driver file is identical to current rpi-6.18.y; rpi-6.12.y differs only in API adaptations)
Firmware: start4.elf 1.20260915

Please note: I found this on a custom image (mainline kernel + the Raspberry Pi media drivers listed above), not on stock Raspberry Pi OS, and I have not been able to re-test it there. I'm reporting it as a possible bug because the driver code involved is identical to rpi-6.18.y — it would be great if someone could check whether it reproduces on Raspberry Pi OS.

Describe the bug

If a client stops both queues of the H.264 decoder (/dev/video10) and then runs the Initialization sequence again on the same file handle with a new stream, the decoder accepts and returns the OUTPUT buffers but never signals V4L2_EVENT_SOURCE_CHANGE. The client waits for the event forever; no CAPTURE buffers, no error.

This is what upstream Chromium's V4L2StatefulVideoDecoder does on every MSE track switch (e.g. a YouTube quality change): Initialize() is called again, which does STREAMOFF on OUTPUT and CAPTURE, sets up a new OUTPUT queue and waits for the source change. The visible result is a frozen last frame of the previous track.

Context: found while bringing up Chromium on a Pi 4 based system (a small hobby OS that uses this driver).

Why this looks like a driver bug

The stateful decoder interface (Documentation/userspace-api/media/v4l/dev-decoder.rst) allows this sequence:

  • "Decoding", last paragraph: the client may "reinitialize the instance by stopping streaming on both queues, releasing all buffers and performing the Initialization sequence again."
  • "Initialization", step 1: once the decoder has parsed the stream, it updates the CAPTURE format and signals a source change event "regardless of whether they match the values set by the client or not."

So after a full stop and a new Initialization the client is entitled to a V4L2_EVENT_SOURCE_CHANGE, also when the new stream has the same resolution as the old one.

Steps to reproduce

Three Annex-B H.264 elementary streams of different size (e.g. 640x360, 1920x1080, 1280x720), played one after another on one file handle:

  1. open("/dev/video10"), VIDIOC_SUBSCRIBE_EVENT(V4L2_EVENT_SOURCE_CHANGE)
  2. Initialization: S_FMT(OUTPUT, H264, no size), REQBUFS(OUTPUT, n), STREAMON(OUTPUT), queue coded data
  3. On SOURCE_CHANGE: Capture Setup (G_FMT(CAPTURE), REQBUFS(CAPTURE), STREAMON(CAPTURE)), decode the stream to its end
  4. Optional drain: V4L2_DEC_CMD_STOP, wait for the V4L2_BUF_FLAG_LAST buffer, V4L2_DEC_CMD_START
  5. STREAMOFF(OUTPUT), STREAMOFF(CAPTURE), REQBUFS(OUTPUT, 0), REQBUFS(CAPTURE, 0)
  6. Back to step 2 with the next stream

A small test program (about one page) that does exactly this is available on request. Usage: v4l2-switch /dev/video10 a.h264 b.h264 c.h264 (options: nodrain, capfirst = STREAMOFF CAPTURE first, reopen = new handle per stream).

Expected behaviour

Each stream produces a SOURCE_CHANGE event and decoded frames.

Actual behaviour

Only the first stream decodes. From the second stream on, OUTPUT buffers are consumed and returned, but the firmware never sends MMAL_EVENT_FORMAT_CHANGED, so handle_fmt_changed() never runs and no event is queued.

The result is the same with or without the drain in step 4 and with either STREAMOFF order. The same streams decode fine when each one gets a newly opened file handle (reopen), and a single stream that changes resolution in the middle also works. So it is specific to restarting on the same handle.

Cause (as far as I can tell)

When the second queue stops, bcm2835_codec_stop_streaming() only disables the MMAL component (vchiq_mmal_component_disable()). The next start_streaming() enables the same component again. The firmware's video_decode seems to keep its stream state across disable/enable: it decodes nothing further and reports no format change for the new stream. Committing the input port format again (S_FMT(OUTPUT) with an existing component goes through vchiq_mmal_port_set_format()) does not reset it either. A freshly created component, as after open(), behaves correctly.

Proposed fix

When the second queue of a decoder stops, release the component and let the next queue_setup() (which already creates it if missing) or start_streaming() create a new one. Also reset the CAPTURE format to the open() default: a new component only reports a format change when the stream differs from the port's format, so a switch to a stream of the same size (360p -> 360p) would otherwise still stall.

I'm not sure this is the preferred place to fix it. If the firmware has a lighter way to reset video_decode (a port-level action instead of tearing down the component), that would be better, and I'm happy to test it. The patch below applies to rpi-6.18.y and rpi-6.12.y.

--- a/drivers/staging/vc04_services/bcm2835-codec/bcm2835-v4l2-codec.c
+++ b/drivers/staging/vc04_services/bcm2835-codec/bcm2835-v4l2-codec.c
@@ -3046,12 +3046,26 @@
 	struct bcm2835_codec_ctx *ctx = vb2_get_drv_priv(q);
 	struct bcm2835_codec_dev *dev = ctx->dev;
 	struct bcm2835_codec_q_data *q_data = get_q_data(ctx, q->type);
-	struct vchiq_mmal_port *port = get_port_data(ctx, q->type);
+	struct vchiq_mmal_port *port;
+	struct vb2_v4l2_buffer *vbuf;
 	int ret = 0;
 
 	v4l2_dbg(1, debug, &ctx->dev->v4l2_dev, "%s: type: %d count %d\n",
 		 __func__, q->type, count);
 	q_data->sequence = 0;
+
+	/* released when both queues stopped (bcm2835_codec_stop_streaming) */
+	if (!ctx->component) {
+		ret = bcm2835_codec_create_component(ctx);
+		if (ret) {
+			while ((vbuf = V4L2_TYPE_IS_OUTPUT(q->type) ?
+					v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx) :
+					v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx)))
+				v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_QUEUED);
+			return ret;
+		}
+	}
+	port = get_port_data(ctx, q->type);
 
 	if (!ctx->component_enabled) {
 		ret = vchiq_mmal_component_enable(dev->instance,
@@ -3216,6 +3230,51 @@
 		ctx->component_enabled = false;
 	}
 
+	/*
+	 * Decoder with both queues stopped: the client starts over, e.g.
+	 * Chromium on every MSE track switch (STREAMOFF of both queues, a new
+	 * OUTPUT queue, then it waits for V4L2_EVENT_SOURCE_CHANGE). The
+	 * firmware's video_decode keeps its stream state across a disable and
+	 * enable of the component and sends no MMAL_EVENT_FORMAT_CHANGED for
+	 * the next stream, so that event never came and the client stalled.
+	 * Release the component; the next REQBUFS or STREAMON creates a fresh
+	 * one, as after open(). The CAPTURE size goes back to the default too:
+	 * a fresh component reports a format change only when the stream
+	 * differs from it, and the next stream may have the same size.
+	 */
+	if (dev->role == DECODE &&
+	    !vb2_is_streaming(v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
+					      V4L2_TYPE_IS_OUTPUT(q->type) ?
+					      V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE :
+					      V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE))) {
+		struct bcm2835_codec_q_data *dst = &ctx->q_data[V4L2_M2M_DST];
+
+		/* still enabled for events if CAPTURE stopped first */
+		if (ctx->component->output[0].enabled)
+			vchiq_mmal_port_disable(dev->instance,
+						&ctx->component->output[0]);
+		if (ctx->component_enabled) {
+			vchiq_mmal_component_disable(dev->instance,
+						     ctx->component);
+			ctx->component_enabled = false;
+		}
+		vchiq_mmal_component_finalise(dev->instance, ctx->component);
+		ctx->component = NULL;
+		dst->crop_width = DEFAULT_WIDTH;
+		dst->crop_height = DEFAULT_HEIGHT;
+		dst->height = DEFAULT_HEIGHT;
+		dst->selection_set = false;
+		dst->bytesperline = get_bytesperline(DEFAULT_WIDTH,
+						     DEFAULT_HEIGHT, dst->fmt,
+						     dev->role);
+		dst->sizeimage = get_sizeimage(dst->bytesperline,
+					       dst->crop_width, dst->height,
+					       dst->fmt);
+		v4l2_dbg(1, debug, &ctx->dev->v4l2_dev,
+			 "%s: both queues stopped, component released\n",
+			 __func__);
+	}
+
 	if (V4L2_TYPE_IS_OUTPUT(q->type))
 		mmal_vchi_buffer_cleanup(&q_data->eos_buffer.mmal);
 

Test results

With the patch, on a Pi 4 (custom image as above, not stock Raspberry Pi OS):

  • test program: 1 of 3 streams decoded before, 8 of 8 after (360p -> 1080p -> 720p, same size 360p -> 360p, with and without drain, both STREAMOFF orders, resolution change inside one stream)
  • Chromium (V4L2 stateful decoder): six quality switches in a row on an MSE test page without a stall; YouTube 1080p H.264 plays through /dev/video10
  • a client that keeps one stream running (no full stop) is unaffected: 120/120 frames decoded in hardware

Related, but not the same

  • #7691: V4L2_DEC_CMD_START does not resume after a source change (resume path, not a full stop)
  • #7674: race between last_buffer_dequeued and the queued SOURCE_CHANGE event
  • #5059: lost frames / incomplete draining on caps renegotiation
  • #3390 (closed): MSE resolution change with GStreamer; a decoder "waiting for something" was mentioned there too, but GStreamer takes a different path
  • PR #7713 (flush signalling) and PR #7733 (EOS before STREAMON) touch other parts of the stop/start code and address different problems
Vorherrschende Sprache
C
Sterne
13.3k
Forks
5.5k
Ø Merge
4 T. 10 Std.
Gemergte PRs (30 T.)
31

Entwicklungsumgebung

Dieses Projekt bietet weder Dev-Container noch Dockerfile noch Beitragsleitfaden – die Einrichtung liegt bei Ihnen. Beginnen Sie mit der README; die allgemeinen Schritte stehen in unserem Leitfaden für den ersten Beitrag.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus raspberrypi/linux

Alle Issues in raspberrypi/linux

Ähnliche Issues

Weitere Issues zu C

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.