Appending a regular command list to an idle immediate command list aborts in LinearStream::getSpace when a UC tag update is pending
Valutazione
- Difficoltà
- 2/5
- Tempo stimato
- 1-3 ore
- Idoneità per principianti
- 20/100
Direzione di ricerca
Start in shared/source/command_stream/command_stream_receiver_hw_base.inl, where flushTaskHeapful emits the UC tag update, and compare it with the nearby barrier handling and the BCS path described in the issue. Add or inspect the ULT for appending a regular list to an idle in-order immediate list with a pending tag update; done means the tag update uses the epilogue stream when present and the reproducer no longer aborts. A linked pull request is already open.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
Appending a regular command list to an in-order immediate command list with zeCommandListImmediateAppendCommandListsExp aborts the process if the immediate list is idle and the CSR has a pending UC tag update:
Abort was called at 90 line in file:
.../shared/source/command_stream/linear_stream.h
(line 91 on master)
NEO::abortUnrecoverable(int, char const*)
NEO::MemorySynchronizationCommands<NEO::Gen12LpFamily>::addBarrierWithPostSyncOperation(...)
NEO::CommandStreamReceiverHw<NEO::Gen12LpFamily>::emitTagUpdateWithoutDCFlush(NEO::LinearStream&)
NEO::CommandStreamReceiverHw<NEO::Gen12LpFamily>::flushTaskHeapful(...)
NEO::CommandStreamReceiverHw<NEO::Gen12LpFamily>::flushTask(...)
L0::CommandListCoreFamilyImmediate<(GFXCORE_FAMILY)18>::flushRegularTask(...)
L0::CommandListCoreFamilyImmediate<(GFXCORE_FAMILY)18>::executeCommandListImmediateWithFlushTaskImpl(...)
L0::CommandListCoreFamilyImmediate<(GFXCORE_FAMILY)18>::flushImmediate(...)
L0::CommandListCoreFamilyImmediate<(GFXCORE_FAMILY)18>::appendCommandLists(...)
L0::zeCommandListImmediateAppendCommandListsExp(...)
It shows up as a sporadic abort in applications that submit work in batches of regular command lists. The reproducer below makes it deterministic.
Cause
Two features meet here:
- When
appendCommandListshas nothing to dispatch from the queue (the immediate list is idle, so there is no wait or implicit dependency), and the command list's batch buffer is dispatched as primary, the queue submits the regular list's first command buffer directly. It wraps that buffer infirstCmdListStreamand marks it full (cmdqueue_hw.inl#L1567-L1572):
The immediate list's own stream is passed along asthis->firstCmdListStream.replaceBuffer(cmdListFirstCmdBuffer->getUnderlyingBuffer(), firstCmdBufferAlignedSize); this->firstCmdListStream.getSpace(firstCmdBufferAlignedSize); this->startingCmdBuffer = &this->firstCmdListStream;dispatchFlags.optionalEpilogueCmdStream, for the commands that follow the task. - Since c78c1515de / 64b79723cc ("reuse cmd buffer without dc flush", 25.44),
ensureCommandBufferAllocationsetsucResourceRequiresTagUpdatewhenever the CSR's command buffer is replaced (command_stream_receiver.cpp#L372). The nextflushTaskHeapfulthen emits the tag update into the task stream (command_stream_receiver_hw_base.inl#L469-L471):if (this->ucResourceRequiresTagUpdate) { this->emitTagUpdateWithoutDCFlush(commandStreamTask); }
When the flush after a CSR command buffer replacement is such a direct submission, the PIPE_CONTROL goes into the full firstCmdListStream, and LinearStream::getSpace hits UNRECOVERABLE_IF(sizeUsed + size > maxAvailableSpace).
I confirmed this with a debugger: at the abort, the task stream passed to flushTaskHeapful has getUsed() == getMaxAvailableSpace() (312 of 312 bytes, i.e. a small regular list's buffer), and CommandStreamReceiver::releaseCommandBufferAllocation ran during the preceding flush.
The other epilogue commands already account for this. The barrier with post-sync a few lines further down in flushTaskHeapful (#L481) and the UC tag update in flushBcsTask (#L228-L232) both write to optionalEpilogueCmdStream when one is set. Only the UC tag update in flushTaskHeapful uses the task stream unconditionally.
Reproducer
repro.c
// Appending a regular command list to an idle in-order immediate command list aborts in
// LinearStream::getSpace once the CSR has a pending UC tag update.
//
// cc -O2 -o repro repro.c -I<level-zero include> -L<loader dir> -lze_loader
// ./repro [iterations]
#include <level_zero/ze_api.h>
#include <stdio.h>
#include <stdlib.h>
#define CHECK(call) \
do { \
ze_result_t res_ = (call); \
if (res_ != ZE_RESULT_SUCCESS) { \
fprintf(stderr, "%s failed: 0x%x (line %d)\n", #call, res_, \
__LINE__); \
exit(1); \
} \
} while (0)
int main(int argc, char **argv) {
int iterations = argc > 1 ? atoi(argv[1]) : 10000;
CHECK(zeInit(0));
uint32_t count = 1;
ze_driver_handle_t driver;
CHECK(zeDriverGet(&count, &driver));
count = 1;
ze_device_handle_t device;
CHECK(zeDeviceGet(driver, &count, &device));
ze_device_properties_t props = {ZE_STRUCTURE_TYPE_DEVICE_PROPERTIES};
CHECK(zeDeviceGetProperties(device, &props));
printf("device: %s\n", props.name);
ze_context_desc_t context_desc = {ZE_STRUCTURE_TYPE_CONTEXT_DESC};
ze_context_handle_t context;
CHECK(zeContextCreate(driver, &context_desc, &context));
// compute engine group
uint32_t ngroups = 0;
CHECK(zeDeviceGetCommandQueueGroupProperties(device, &ngroups, NULL));
ze_command_queue_group_properties_t groups[16];
for (uint32_t i = 0; i < ngroups && i < 16; i++)
groups[i] = (ze_command_queue_group_properties_t){
ZE_STRUCTURE_TYPE_COMMAND_QUEUE_GROUP_PROPERTIES};
CHECK(zeDeviceGetCommandQueueGroupProperties(device, &ngroups, groups));
uint32_t ordinal = 0;
for (uint32_t i = 0; i < ngroups; i++)
if (groups[i].flags & ZE_COMMAND_QUEUE_GROUP_PROPERTY_FLAG_COMPUTE) {
ordinal = i;
break;
}
ze_command_queue_desc_t queue_desc = {ZE_STRUCTURE_TYPE_COMMAND_QUEUE_DESC};
queue_desc.ordinal = ordinal;
queue_desc.flags = ZE_COMMAND_QUEUE_FLAG_IN_ORDER;
queue_desc.mode = ZE_COMMAND_QUEUE_MODE_ASYNCHRONOUS;
ze_command_list_handle_t immediate;
CHECK(zeCommandListCreateImmediate(context, device, &queue_desc, &immediate));
ze_command_list_desc_t list_desc = {ZE_STRUCTURE_TYPE_COMMAND_LIST_DESC};
list_desc.commandQueueGroupOrdinal = ordinal;
list_desc.flags = ZE_COMMAND_LIST_FLAG_IN_ORDER;
ze_command_list_handle_t regular;
CHECK(zeCommandListCreate(context, device, &list_desc, ®ular));
const size_t size = 4096;
ze_device_mem_alloc_desc_t mem_desc = {ZE_STRUCTURE_TYPE_DEVICE_MEM_ALLOC_DESC};
void *a, *b;
CHECK(zeMemAllocDevice(context, &mem_desc, size, 64, device, &a));
CHECK(zeMemAllocDevice(context, &mem_desc, size, 64, device, &b));
for (int i = 0; i < iterations; i++) {
// work on the immediate list, which consumes space in the CSR's command buffer;
// when that buffer is replaced, the CSR schedules a UC tag update for its next flush
CHECK(zeCommandListAppendMemoryCopy(immediate, b, a, size, NULL, 0, NULL));
CHECK(zeCommandListHostSynchronize(immediate, UINT64_MAX));
// append a regular command list to the now idle immediate list: with nothing to
// dispatch from the queue, the regular list's own buffer becomes the batch buffer
CHECK(zeCommandListReset(regular));
for (int j = 0; j < 4; j++) {
uint32_t pattern = i + j;
CHECK(zeCommandListAppendMemoryFill(regular, a, &pattern, sizeof(pattern),
size, NULL, 0, NULL));
}
CHECK(zeCommandListClose(regular));
CHECK(zeCommandListImmediateAppendCommandListsExp(immediate, 1, ®ular, NULL,
0, NULL));
CHECK(zeCommandListHostSynchronize(immediate, UINT64_MAX));
if ((i + 1) % 1000 == 0)
printf("%d iterations\n", i + 1);
}
printf("done\n");
CHECK(zeMemFree(context, a));
CHECK(zeMemFree(context, b));
CHECK(zeCommandListDestroy(regular));
CHECK(zeCommandListDestroy(immediate));
CHECK(zeContextDestroy(context));
return 0;
}
$ cc -O2 -o repro repro.c -lze_loader
$ ./repro 20000
device: Intel(R) Iris(R) Xe Graphics
Abort was called at 90 line in file:
/workspace/srcdir/compute-runtime/shared/source/command_stream/linear_stream.h
Aborted
It aborts within the first thousand iterations, on every run. The memory copy on the immediate list is there to consume CSR command buffer space, so that the buffer is eventually replaced. The host synchronization before the append makes the immediate list idle, which selects the direct submission of the regular list.
Environment
- Intel Iris Xe Graphics (Tiger Lake GT2, 8086:9a49), i915, Linux 6.12 (Debian 13)
- compute-runtime 26.35.39758.10 (release build), IGC 2.41.5, gmmlib 22.10.0
- The code involved is unchanged on master (7582a08ed2).
- 25.18 (LTS) is not affected, since it predates the UC tag update.
Fix
Emit the tag update into the epilogue stream when there is one, as the barrier and the BCS path already do. I'll open a PR with that change and a ULT.
Downstream, this crashes the oneAPI.jl test suite, which submits kernel launches in batches of regular command lists (JuliaGPU/oneAPI.jl#693, failures seen on JuliaGPU/oneAPI.jl#707).
- Lingua principale
- C++
- Stelle
- 1.5k
- Fork
- 303
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Nessun modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di intel/compute-runtime
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
intel/compute-runtime#943 ·
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 12/100
intel/compute-runtime#1026 ·
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 15/100
intel/compute-runtime#1024 ·
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
intel/compute-runtime#1019 ·
-
OS: Linux Type: Bug
Difficoltà 4/5 3-5 giorni Idoneità per principianti 34/100
intel/compute-runtime#1017 ·
Tutte le issue di intel/compute-runtime
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 83/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
-
[BUG] Clipboard History: Buffer over-read in WideCharToMultiByte on double-click pasteForse già presa @softmgr l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
notepad-plus-plus/notepad-plus-plus#18519 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
justcallmekoko/ESP32Marauder#1635 ·
I maintainer di solito rispondono entro 1 giorno
-
3.7.6-dev commit 80481ea - files not centered when resolution differs and zoom = "pixel-to-pixel"Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
grizzlypeak3d/DJV#931 ·