[DLPack] Update stream=None default guideline
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Documentation
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python, pytorch
- Domain
- documentation
Research direction
No file or test is named. Start by reviewing the DLPack stream=None guideline and the CUDA graph example in this issue, then resolve the default-behavior discussion and document explicit stream passing, no-sync behavior, and the rationale; done means the guideline and rationale are updated consistently.
Written by the indexing model from the issue text.
Description
Previously we landed stream=None mapping to legacy default stream (a safer case). As DLPack get popularized, one most canonical use-case is to exchange between library and pytorch. As most libraries are not updated to take stream passing, and many expects that the behavior is no-sync, which works better for cases like CUDAGraph:
s = torch.cuda.Stream()
x = torch.randn(8, device="cuda")
g = torch.cuda.CUDAGraph()
with torch.cuda.stream(s):
with torch.cuda.graph(g):
_ = x + 1
mylib_tensor = mylib.from_dlpack(x)
mylib_kernel(mylib_tensor)
In the above code example, if the stream=None maps to no sync(currently stream=-1), then the cuda graph capture will work out of box. Otherwise, the cudagraph capture no longer work because of the sync. This is only the choice of default behavior as mylib can always pick a specific stream to be passed in.
So the discussion only focuses on the guideline for default behavior. The original rationale of the default was that legacy stream was a "safe choice". However, as DLPack based exchange becomes popularized and CUDAGraph integration becomes criticial. It could make sense for the default to optimize for common usecases (stream=None default to nosync if applicable).
It is worth pointing out the nosync was also the implicit original behavior before the stream proposal before frameworks get updated (many only recently like in the case of torch), so many libraries may indeed implicitly relied on such behavior.
Regardless of choices here, I think we should definitely update guideline to encourage the users to explicitly pass in stream, and document the rationale of nosync behavior, relation to CUDAgraph etc, to help libraries pick.
- Dominant language
- Python
- Stars
- 281
- Forks
- 52
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from data-apis/array-api
-
bug Maintenance Narrative Content
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Maintenance
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
All issues in data-apis/array-api
Similar issues
-
essnmx good first issue
Difficulty 1/5 Under an hour Newbie friendliness 95/100
-
[Feature] 奇物选择添加优先级 Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
syfoud/Simulated_Scepter#174 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Giskard-AI/giskard-oss#2840 · 1 comment ·
-
A claim comment carrying the issue number is silently declined while the workflow reports success Openarea: repo bug perceived difficulty: 2
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
yeti-platform/yeti#1380 ·