Prometheus crashes with SIGSEGV when accessing mapped chunk during PromQL query

What did you do?

After Prometheus started up, I simulated a disk I/O hang on Prometheus's storage for about 60 seconds. Immediately after the storage became responsive again, I sent PromQL queries.

What did you expect to see?

Prometheus should validate that a chunk reference is valid before attempting to access it. If the reference is invalid, it should fail gracefully by returning an error or logging a warning — not crash with a SIGSEGV.

What did you see instead? Under which circumstances?

N/A

System information

No response

Prometheus version

v2.55.1

Prometheus configuration file

Alertmanager version

Alertmanager configuration file

Logs

unexpected fault address 0x7f833cf2dab8
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x7f833cf2dab8 pc=0x55c4f63cc9fd]
goroutine 660243 gp=0xc03520a540 m=24 mp=0xc000f80008 [running]:
runtime.throw({0x55c4f679ffbd?, 0x0?})
	runtime/panic.go:1101 +0x4a fp=0xc0136a32d0 sp=0xc0136a32a0 pc=0x55c4f3cc306a
runtime.sigpanic()
	runtime/signal_unix.go:939 +0x26c fp=0xc0136a3330 sp=0xc0136a32d0 pc=0x55c4f3cc522c
github.com/prometheus/prometheus/tsdb/chunks.(*ChunkDiskMapper).Chunk(0xc000eb82d0, 0x20332daa0)
	github.com/prometheus/prometheus/tsdb/chunks/head_chunks.go:753 +0x47d fp=0xc0136a34f0 sp=0xc0136a3330 pc=0x55c4f63cc9fd
github.com/prometheus/prometheus/tsdb.(*memSeries).chunk(0xc0076536b0, 0x10000000002?, 0x7f8341d8c588?, 0xc00066a2d8)
	github.com/prometheus/prometheus/tsdb/head_read.go:462 +0x95 fp=0xc0136a3548 sp=0xc0136a34f0 pc=0x55c4f66a4955
github.com/prometheus/prometheus/tsdb.(*Head).chunkFromSeries(0xc00066a008, 0xc0076536b0, 0x3, 0x10?, 0x1967e683240, 0x1967e6f8540, 0xc000a042c0, 0x0)
	github.com/prometheus/prometheus/tsdb/head_read.go:393 +0x9d fp=0xc0136a3630 sp=0xc0136a3548 pc=0x55c4f66a43fd
github.com/prometheus/prometheus/tsdb.(*headChunkReader).chunk(0xc014c171c0, {0x1619b000003, {0x0, 0x0}, 0x1967e4eec51, 0x1967e69ee61}, 0x0)
	github.com/prometheus/prometheus/tsdb/head_read.go:379 +0x149 fp=0xc0136a36d8 sp=0xc0136a3630 pc=0x55c4f66a41e9
github.com/prometheus/prometheus/tsdb.(*headChunkReader).ChunkOrIterable(0x0?, {0x1619b000003, {0x0, 0x0}, 0x1967e4eec51, 0x1967e69ee61})
	github.com/prometheus/prometheus/tsdb/head_read.go:350 +0x3e fp=0xc0136a3720 sp=0xc0136a36d8 pc=0x55c4f66a3f9e
github.com/prometheus/prometheus/tsdb.(*populateWithDelGenericSeriesIterator).next(0xc01e7ec7e0, 0x0)
	github.com/prometheus/prometheus/tsdb/querier.go:655 +0x308 fp=0xc0136a3818 sp=0xc0136a3720 pc=0x55c4f66bc2a8
github.com/prometheus/prometheus/tsdb.(*populateWithDelSeriesIterator).Next(0xc01e7ec7e0)
	github.com/prometheus/prometheus/tsdb/querier.go:735 +0x4f fp=0xc0136a3840 sp=0xc0136a3818 pc=0x55c4f66bca0f
github.com/prometheus/prometheus/storage.(*MemoizedSeriesIterator).Reset(...)
	github.com/prometheus/prometheus/storage/memoized_iterator.go:62
github.com/prometheus/prometheus/promql.(*evaluator).evalSeries(0xc000c5e460, {0x55c4f8b79b70, 0xc00fb0d290}, {0xc008565408, 0x15c, 0x0?}, 0x0, 0x0)
	github.com/prometheus/prometheus/promql/engine.go:1453 +0x285 fp=0xc0136a39c8 sp=0xc0136a3840 pc=0x55c4f6414b85
github.com/prometheus/prometheus/promql.(*evaluator).eval(0xc000c5e460, {0x55c4f8b79b70, 0xc00fb0d1d0}, {0x55c4f8b7dfc0, 0xc003ddcc60})
	github.com/prometheus/prometheus/promql/engine.go:1949 +0x1725 fp=0xc0136a4358 sp=0xc0136a39c8 pc=0x55c4f6416e85
github.com/prometheus/prometheus/promql.(*evaluator).Eval(0xc000c5e460, {0x55c4f8b79b70?, 0xc00fb0d1d0?}, {0x55c4f8b7dfc0?, 0xc003ddcc60?})
	github.com/prometheus/prometheus/promql/engine.go:1104 +0xaf fp=0xc0136a43f8 sp=0xc0136a4358 pc=0x55c4f6410e2f
github.com/prometheus/prometheus/promql.(*Engine).execEvalStmt(0xc000c96900, {0x55c4f8b79b70, 0xc00fb0cf90}, 0xc000c5e2a0, 0xc00f67ca50)
	github.com/prometheus/prometheus/promql/engine.go:799 +0xbeb fp=0xc0136a4680 sp=0xc0136a43f8 pc=0x55c4f640e7ab
github.com/prometheus/prometheus/promql.(*Engine).exec(0xc000c96900, {0x55c4f8b79b70, 0xc00fb0cf90}, 0xc000c5e2a0)
	github.com/prometheus/prometheus/promql/engine.go:678 +0x41a fp=0xc0136a4880 sp=0xc0136a4680 pc=0x55c4f640cb9a
github.com/prometheus/prometheus/promql.(*query).Exec(0xc000c5e2a0, {0x55c4f8b79b70, 0xc00fb0cde0})
	github.com/prometheus/prometheus/promql/engine.go:245 +0x1c5 fp=0xc0136a4970 sp=0xc0136a4880 pc=0x55c4f640a485
github.com/prometheus/prometheus/web/api/v1.(*API).queryRange(0xc00111c840, 0xc00999d900)
	github.com/prometheus/prometheus/web/api/v1/api.go:562 +0x1146