Work behind a long event wait on an immediate command list is silently dropped (DG2, i915); host sync then hangs or reports completion
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức phù hợp với người mới
- 35/100
- Loại issue
- Lỗi
- Độ rõ ràng
- Khá rõ ràng
- Mức độ hoạt động
- Sôi nổi
- Lĩnh vực
- backend, devtools, operating-systems
Hướng nghiên cứu
Start by comparing the reported i915 fence timeout with the immediate-list submission and synchronization paths in the compute-runtime sources; the issue does not name those files. Build and run the supplied hostsync.c reproducer on an Arc A770 with i915, especially variants 3 and 6. Done means work resumes after the event is signaled, or synchronization reports a device error rather than losing work, hanging, or reporting false completion.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
On an Arc A770 (DG2) with the i915 kernel driver, work appended to an in-order immediate command list behind a wait on an event is silently dropped if the event stays unsignaled for more than about 10 seconds. Signaling the event afterwards doesn't run it. zeCommandListHostSynchronize on that list then either never returns, or returns ZE_RESULT_SUCCESS although the work never executed. The second case happens once another immediate list on the same engine submits work and synchronizes.
Each run logs two i915 messages, about 10 s after the work is blocked:
Fence expiration time out i915-0000:2f:00.0:hostsync[283317]:a!
Fence expiration time out i915-0000:2f:00.0:hostsync[283317]:8!
That matches CONFIG_DRM_I915_FENCE_TIMEOUT=10000 in this kernel. My guess is that each append is submitted as its own execbuf that implicitly waits for the previous, blocked one, and that i915 runs it anyway with an error once that wait times out, skipping its payload. I haven't verified that in the i915 sources.
Reproducer
hostsync.c below:
- Creates an in-order asynchronous immediate list S on ordinal 0.
- Appends a wait on an unsignaled host-visible event, then a fill of a host allocation with 42.
- Optionally does something else (selected with a command-line argument).
- Checks S with
zeCommandListHostSynchronize(S, 0). - Signals the event and synchronizes S again.
Build with cc -o hostsync hostsync.c -lze_loader.
| Variant | Between blocking S and re-checking it | Result |
|---|---|---|
| 0 | nothing | correct: NOT_READY, then 42 after signaling |
| 1, 2 | create a second immediate list T; also sync it while empty (2) | correct |
| 3 | T submits a fill and is synchronized | T's sync takes ~20 s (two fence expirations). Then S reports SUCCESS with the fill not executed, and signaling never runs it. |
| 5 | as 3, but S is busy with 200 large fills instead of waiting | correct: T's sync waits for S |
| 6 | no T, sleep 25 s | S correctly stays NOT_READY, but after signaling the fill never runs and zeCommandListHostSynchronize(S, UINT64_MAX) hangs forever |
| 7 | as 3, with a signal event on S's fill | the event and S correctly stay NOT_READY, but after signaling the work is gone and synchronizing hangs |
Variant 0, then variant 3:
device: Intel(R) Arc(TM) A770 Graphics, variant 0
S gated: NOT_READY, buf=0
S re-checked: NOT_READY, buf=0
after signaling: buf=42
device: Intel(R) Arc(TM) A770 Graphics, variant 3
S gated: NOT_READY, buf=0
S re-checked: SUCCESS, buf=0 <-- BUG: reported done, work not executed
after signaling: buf=0
With NEO_LogWaitingForCompletion=1, variant 3 shows that all lists share one CSR tag. S waits for task count 4, and T's submission moves the tag to 7, although S's work behind the wait never ran:
Waiting for task count 4 at location 0x7efe86a52000 with timeout 0. Current value: 2
Waiting completed. Current value: 2
S gated: NOT_READY, buf=0
Waiting for task count 6 at location 0x7efe86a52000 with timeout 0. Current value: 2
Waiting completed. Current value: 7
Waiting for task count 4 at location 0x7efe86a52000 with timeout 0. Current value: 7
Waiting completed. Current value: 7
S re-checked: SUCCESS, buf=0 <-- BUG: reported done, work not executed
Expected: work waiting on an event runs once the event is signaled, however long that takes. If the driver can't guarantee that, it should at least report an error (e.g. ZE_RESULT_ERROR_DEVICE_LOST) rather than drop the work, hang, or report a list as complete when it isn't.
This may be related to #904, which also involves synchronization after ~20 s on an A770.
hostsync.c
// zeCommandListHostSynchronize reports an in-order immediate command list as done while
// work on it is still blocked waiting for an unsignaled event.
//
// usage: hostsync [variant]
// 0: nothing else happens before re-checking S
// 1: create a second immediate list T
// 2: create T and host-synchronize it (empty)
// 3: create T, append a fill to it and host-synchronize it
// 4: host-synchronize S itself with a 100 ms timeout first (no other list)
// 5: like 3, but S is busy with large fills instead of waiting for an event
// 6: no other list, but wait 25 s (longer than i915's fence timeout) before re-checking
// 7: like 3, and also check an event signaled after S's work
#include <level_zero/ze_api.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>
#define CHECK(x) do { ze_result_t r = (x); if (r != ZE_RESULT_SUCCESS) { \
fprintf(stderr, "%s:%d: %s = 0x%x\n", __FILE__, __LINE__, #x, r); exit(1); } } while (0)
static const char *res(ze_result_t r) {
return r == ZE_RESULT_SUCCESS ? "SUCCESS" : r == ZE_RESULT_NOT_READY ? "NOT_READY" : "other";
}
static ze_command_list_handle_t immediate(ze_context_handle_t ctx, ze_device_handle_t dev) {
ze_command_queue_desc_t desc = {ZE_STRUCTURE_TYPE_COMMAND_QUEUE_DESC};
desc.ordinal = 0;
desc.index = 0;
desc.flags = ZE_COMMAND_QUEUE_FLAG_IN_ORDER;
desc.mode = ZE_COMMAND_QUEUE_MODE_ASYNCHRONOUS;
desc.priority = ZE_COMMAND_QUEUE_PRIORITY_NORMAL;
ze_command_list_handle_t list;
CHECK(zeCommandListCreateImmediate(ctx, dev, &desc, &list));
return list;
}
int main(int argc, char **argv) {
int variant = argc > 1 ? atoi(argv[1]) : 2;
CHECK(zeInit(ZE_INIT_FLAG_GPU_ONLY));
uint32_t n = 1;
ze_driver_handle_t drv;
CHECK(zeDriverGet(&n, &drv));
n = 1;
ze_device_handle_t dev;
CHECK(zeDeviceGet(drv, &n, &dev));
ze_device_properties_t props = {ZE_STRUCTURE_TYPE_DEVICE_PROPERTIES};
CHECK(zeDeviceGetProperties(dev, &props));
ze_context_desc_t cdesc = {ZE_STRUCTURE_TYPE_CONTEXT_DESC};
ze_context_handle_t ctx;
CHECK(zeContextCreate(drv, &cdesc, &ctx));
ze_device_mem_alloc_desc_t ddesc = {ZE_STRUCTURE_TYPE_DEVICE_MEM_ALLOC_DESC};
ze_host_mem_alloc_desc_t hdesc = {ZE_STRUCTURE_TYPE_HOST_MEM_ALLOC_DESC};
volatile uint32_t *buf;
CHECK(zeMemAllocHost(ctx, &hdesc, 64, 64, (void **)&buf));
buf[0] = 0;
ze_event_pool_desc_t pdesc = {ZE_STRUCTURE_TYPE_EVENT_POOL_DESC};
pdesc.flags = ZE_EVENT_POOL_FLAG_HOST_VISIBLE;
pdesc.count = 2;
ze_event_pool_handle_t pool;
CHECK(zeEventPoolCreate(ctx, &pdesc, 1, &dev, &pool));
ze_event_desc_t edesc = {ZE_STRUCTURE_TYPE_EVENT_DESC};
edesc.index = 0;
edesc.signal = ZE_EVENT_SCOPE_FLAG_HOST;
edesc.wait = ZE_EVENT_SCOPE_FLAG_HOST;
ze_event_handle_t gate;
CHECK(zeEventCreate(pool, &edesc, &gate));
ze_event_handle_t done;
edesc.index = 1;
CHECK(zeEventCreate(pool, &edesc, &done));
ze_command_list_handle_t s = immediate(ctx, dev);
// warm up S, so that its first submission isn't special
uint32_t zero = 0;
CHECK(zeCommandListAppendMemoryFill(s, (void *)buf, &zero, 4, 4, NULL, 0, NULL));
CHECK(zeCommandListHostSynchronize(s, UINT64_MAX));
// S: wait for the gate (or do lots of work), then write 42
uint32_t val = 42;
if (variant == 5) {
size_t size = (size_t)1 << 30;
void *big;
CHECK(zeMemAllocDevice(ctx, &ddesc, size, 64, dev, &big));
for (int i = 0; i < 200; i++)
CHECK(zeCommandListAppendMemoryFill(s, big, &i, 4, size, NULL, 0, NULL));
} else {
CHECK(zeCommandListAppendWaitOnEvents(s, 1, &gate));
}
CHECK(zeCommandListAppendMemoryFill(s, (void *)buf, &val, 4, 4,
variant == 7 ? done : NULL, 0, NULL));
printf("device: %s, variant %d\n", props.name, variant);
printf("S gated: %s, buf=%u\n", res(zeCommandListHostSynchronize(s, 0)), buf[0]);
ze_command_list_handle_t t = NULL;
switch (variant) {
case 0:
break;
case 1:
t = immediate(ctx, dev);
break;
case 2:
t = immediate(ctx, dev);
CHECK(zeCommandListHostSynchronize(t, UINT64_MAX));
break;
case 3:
case 5:
case 7: {
void *other;
CHECK(zeMemAllocDevice(ctx, &ddesc, 64, 64, dev, &other));
t = immediate(ctx, dev);
CHECK(zeCommandListAppendMemoryFill(t, other, &zero, 4, 4, NULL, 0, NULL));
CHECK(zeCommandListHostSynchronize(t, UINT64_MAX));
break;
}
case 6:
sleep(25);
break;
case 4:
printf("S 100ms timeout: %s, buf=%u\n",
res(zeCommandListHostSynchronize(s, 100000000)), buf[0]);
break;
}
if (variant == 7)
printf("done event: %s, buf=%u\n", res(zeEventHostSynchronize(done, 0)), buf[0]);
ze_result_t r = zeCommandListHostSynchronize(s, 0);
printf("S re-checked: %s, buf=%u%s\n", res(r), buf[0],
r == ZE_RESULT_SUCCESS && buf[0] != 42 ? " <-- BUG: reported done, work not executed" : "");
CHECK(zeEventHostSignal(gate));
CHECK(zeCommandListHostSynchronize(s, UINT64_MAX));
printf("after signaling: buf=%u\n", buf[0]);
return r == ZE_RESULT_SUCCESS && buf[0] != 42;
}
Environment
- Intel Arc A770 (DG2), i915 kernel driver
- Linux 7.2.11 (Arch),
CONFIG_DRM_I915_FENCE_TIMEOUT=10000 - compute-runtime 26.35.39758.10, level-zero loader 1.32.0
The same thing happens in oneAPI.jl's tests, which ship the same compute-runtime release. I haven't tried the xe kernel driver.
- Ngôn ngữ chính
- C++
- Star
- 1.5k
- Fork
- 304
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Chuẩn bị môi trường
- Không có Dockerfile hay tệp Docker Compose
- Không có mẫu pull request
- Đọc hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của intel/compute-runtime
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
intel/compute-runtime#943 ·
-
OS: Windows Type: Bug
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 45/100
intel/compute-runtime#1032 ·
-
Component: IGC Type: Regression
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 48/100
intel/compute-runtime#1031 · 3 bình luận ·
-
Độ khó 4/5 1-2 ngày Mức phù hợp với người mới 54/100
intel/compute-runtime#1030 ·
-
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 42/100
intel/compute-runtime#1029 ·
Tất cả issue của intel/compute-runtime
Issue tương tự
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 62/100
foxglove/mcap#1907 · 1 bình luận ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 66/100
Maintainer thường phản hồi trong vòng 1 ngày
-
bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 85/100
telegramdesktop/tdesktop#31505 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
Maintainer thường phản hồi trong vòng 1 ngày
-
ll-builder shell completions omit existing lockfile commands and --lockedCó thể đã có người làm @project-nju-learner đã nhận hôm nay. Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 90/100
OpenAtom-Linyaps/linyaps#2291 ·
Maintainer thường phản hồi trong vòng 1 ngày