The Notebook for TT Codec does not match the actual flow to setup codec for cuda
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 68/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- jupyter-notebook, pytorch
- Domain
- documentation, machine-learning
Research direction
Start with the TT Codec notebook's setup code field and compare its device handling with the other notebooks. Verify how the model's device parameter is configured for CUDA or another device, then run the notebook to confirm the input and model weights use the same device without the reported mismatch.
Written by the indexing model from the issue text.
Description
While implementing the codec in my research I noticed, that the notebook does not match the way I actually had to setup the codec. When setting it up the way it was in the notebook I ran into this issue:
RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor
This was frustrating as I though the issue was on my end(it kinda was). I wanted to run the codec on GPU using model.to(device). This put the model into Cuda, but inside the Model itself I noticed all the default values are set to cpu. Making it mandatory to use the device parameter in the model setup when using the model on a different device. Unlike the other Notebooks it is not documented how to use the Codec on CUDA or other devices.
At least the device you setup in code field one should be used somewhere. Thanks a lot for your contribution though
- Dominant language
- Jupyter Notebook
- Stars
- 28
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Similar issues
-
user-reported
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Kong/developer.konghq.com#7316 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
HarperFast/skills#96 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
bug triage:deciding
Difficulty 1/5 Under an hour Newbie friendliness 88/100
open-telemetry/otel-arrow#4132 ·
-
Ecosystem: ClawMetry — the Qwen Code reader is now free and open source (follow-up to #9294 / #9338) Opencategory/integration priority/P3 scope/documentation status/ready-for-human type/feature-request
Difficulty 1/5 Under an hour Newbie friendliness 84/100