More (optional) load time verification
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- python
- Domain
- developer-experience, machine-learning
Research direction
Start by reading the verification logic in src/transformers/integrations/deepgemm.py lines 80-204 and sonicmoe.py lines 49-68, then trace the kernel-loading get-kernel entry point. Define how load-time checks should cover NVCC, CUDA architecture, and optional dependency versions, including whether they are flag-controlled. Done means the supported checks work at load time and the duplicated manual verification can be reduced.
Written by the indexing model from the issue text.
Description
Description
Based on our slack conversation
A lot of kernels still need to be verified after loading, leading to a lot of bloated code
- https://github.com/huggingface/transformers/blob/7c8b1d4459c233295be9d25878b73a1b68f23d59/src/transformers/integrations/deepgemm.py#L80-L204
- https://github.com/huggingface/transformers/blob/7c8b1d4459c233295be9d25878b73a1b68f23d59/src/transformers/integrations/sonicmoe.py#L49-L68
It makes the development and usage way more painful and imo kernels could take a part here to allow on load verification for these certain features attributes
- NVCC
- (Cuda) Architecture
- Cute dsl versioning (so any optional dep)
Motivation
Make kernels as dumb to use as we can :D
Proposed approach
Load time verification on get kernel that checks for hw specifics and pinned versions; maybe hidden behind a flag
Alternatives considered
I mean yes, that is exactly what we currently do -> manually check ourselves
Additional context
No response
- Dominant language
- Python
- Stars
- 783
- Forks
- 143
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 57
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from huggingface/kernels
-
kernels-common: hash_variant loads entire file into memory, causing high RSS on large .so filesOpen
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
huggingface/kernels#956 · 1 comment ·
Maintainers usually reply within 1 day
-
kernels benchmark pulls from the old model repos and says every gpu is cpuPossibly taken @drbh claimed this 9 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
huggingface/kernels#914 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
huggingface/kernels#949 ·
Maintainers usually reply within 1 day
-
Add `kernels describe-api` to the "use layers" cookbookPossibly taken @ariG23498 claimed this 20 days ago. Open
huggingface/kernels#906 · 3 comments · 1 assignee ·
Maintainers usually reply within 1 day
-
Support multiple upstreams (make `upstream` a list)May be free again A pull request for this issue was closed without being merged. Open
Difficulty 4/5 3-5 days Newbie friendliness 58/100
huggingface/kernels#850 ·
Maintainers usually reply within 1 day
All issues in huggingface/kernels
Similar issues
-
needs-area-label
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 2 days
-
enhancement good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Official-Space-AI/OpenPhysGraspGen#114 ·
Maintainers usually reply within 2 days
-
changelog investigate
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
ramnes/notion-sdk-py#441 ·
-
qa_bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NVIDIA-NeMo/Gym#5644 ·
Maintainers usually reply within 2 days
-
BUG: catalog database credentials printed to logs at startupPossibly taken @cjboyle claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day