Assertion "(cuStreamQuery(0)) == (CUDA_ERROR_NOT_READY)"
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
Research direction
Reproduce the gdrcopy_pplat failure and start at the assertion in pplat.cu:257, using the complete log to characterize the failing stream query. Check the reported GPUDirectAsync/libgdsync configuration and nvidia-peermem status; done means the test no longer fails at this assertion.
Written by the indexing model from the issue text.
Description
Running gdrcopy_pplat fails with Assertion "(cuStreamQuery(0)) == (CUDA_ERROR_NOT_READY)" failed at pplat.cu:257.
See complete logs below
Click me
GPU id:0; name: Tesla V100-SXM2-32GB; Bus id: 0001:00:00
GPU id:1; name: Tesla V100-SXM2-32GB; Bus id: 0002:00:00
GPU id:2; name: Tesla V100-SXM2-32GB; Bus id: 0003:00:00
GPU id:3; name: Tesla V100-SXM2-32GB; Bus id: 0004:00:00
GPU id:4; name: Tesla V100-SXM2-32GB; Bus id: 0005:00:00
GPU id:5; name: Tesla V100-SXM2-32GB; Bus id: 0006:00:00
GPU id:6; name: Tesla V100-SXM2-32GB; Bus id: 0007:00:00
GPU id:7; name: Tesla V100-SXM2-32GB; Bus id: 0008:00:00
selecting device 0
device ptr: 0x7f6bb7a00000
gpu alloc fn: cuMemAlloc
map_d_ptr: 0x7f6bdc010000
info.va: 7f6bb7a00000
info.mapped_size: 65536
info.page_size: 65536
info.mapped: 1
info.wc_mapping: 1
page offset: 0
user-space pointer: 0x7f6bdc010000
CPU does gdr_copy_to_mapping and GPU writes back via cuMemHostAlloc'd buffer.
Running 1000 iterations with data size 4 bytes.
Assertion "(cuStreamQuery(0)) == (CUDA_ERROR_NOT_READY)" failed at pplat.cu:257
If useful, I have GPUDirectAsync configured and nvidia-peermem activated.
- Dominant language
- C
- Stars
- 1.4k
- Forks
- 198
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA/gdrcopy
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Proposal: honor `TMPDIR` in gdrcopy package build scriptsPossibly taken @efegokdemir claimed this 53 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 58/100
Similar issues
-
SPIDevice selects an active-high device as soon as it is createdPossibly taken @mikeysklar claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
adafruit/circuitpython#11869 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
gnif/LookingGlass#1352 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
bazel-contrib/rules_go#4784 · 1 comment ·
Maintainers usually reply within 1 day