tokenizer for triton inference server
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 20/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Domain
- machine-learning
Research direction
Start by reviewing the repository's documented tokenizer API and examples, then compare them with the Python AutoTokenizer usage and the all-MiniLM-L6-v2 model named in the issue. Done means establishing whether this model and Triton inference-server workflow are supported, or documenting the missing compatibility requirements.
Written by the indexing model from the issue text.
Description
hi,
Can this be used with triton inference server for huggingface setfit (https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)?
here is what i currently do with python:
from transformers import AutoTokenizer
# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/all-MiniLM-L6-v2')
# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
Thanks,
Gerald
- Dominant language
- C++
- Stars
- 514
- Forks
- 132
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from mlc-ai/tokenizers-cpp
-
Missing Cargo.lockOpen
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
mlc-ai/tokenizers-cpp#96 · 1 reaction ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
mlc-ai/tokenizers-cpp#88 · 2 reactions ·
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
mlc-ai/tokenizers-cpp#95 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
mlc-ai/tokenizers-cpp#94 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
mlc-ai/tokenizers-cpp#91 ·
All issues in mlc-ai/tokenizers-cpp
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
lldb
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
llvm/llvm-project#229592 · 11 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 3 days
-
Difficulty 1/5 Under an hour Newbie friendliness 94/100
llvm/offload-test-suite#1560 ·
Maintainers usually reply within 1 day
-
agent:Windows bug MEDIUM performance tooling
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day