attention_mask is discarded in fmha_interface_wrapper: is a varlen or padding-mask prefill kernel wanted?
Maintainers usually reply within 8 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Active
- Tech stack
- python
- Domain
- machine-learning, performance
Research direction
Start in src/tilegym/ops/attn_interface.py at fmha_interface_wrapper and inspect how attention_mask is received and how fmha handles dense causal input. Read the SWA prefill work in #107 and compare it with the roadmap items named in the issue. The next step is to establish an agreed scope and design; implementation is not yet defined.
Written by the indexing model from the issue text.
Description
fmha_interface_wrapper in src/tilegym/ops/attn_interface.py accepts attention_mask from the HF attention interface but never passes it on; fmha itself only supports dense causal input. So padded batches at batch_size > 1 attend over padding tokens on the patched path.
Is a varlen-style (cu_seqlens packed, FlashAttention-varlen-like) or padding-mask-aware prefill kernel something you would want as an external contribution to ops/cutile/experimental/, in the style of the SWA prefill from #107? Or is this territory already covered by the roadmap's "Autoregressive Flash Attention (WIP Internal)" or the planned Flex Attention work?
Asking for scope before writing anything; happy to write up a concrete design in this thread if it is wanted.
- Dominant language
- Python
- Stars
- 830
- Forks
- 92
- Avg merge
- 8d 11h
- Merged PRs (30d)
- 8
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA/TileGym
-
Difficulty 5/5 Over a week Newbie friendliness 12/100
Maintainers usually reply within 8 days
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
NVIDIA/TileGym#210 · 1 comment ·
Maintainers usually reply within 8 days
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
NVIDIA/TileGym#195 · 2 comments ·
Maintainers usually reply within 8 days
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
NVIDIA/TileGym#190 · 2 comments ·
Maintainers usually reply within 8 days
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
NVIDIA/TileGym#169 · 5 comments · 1 reaction ·
Maintainers usually reply within 8 days
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
earthtojake/text-to-cad#758 ·
Maintainers usually reply within 1 day
-
domain:upgrades enhancement needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
gke-labs/kube-agents#4634 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
agardnerIT/opamp-server-py#116 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
enhancement good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
elviskahoro/sdk-python-attio#144 · 1 comment ·
Maintainers usually reply within 1 day