Feat: Add tokens counter
Assessment
This issue has not been assessed yet.
Description
Overview
It would be useful to add the ability for neural to get the token count for some given input. This would help prevent initiating requests that accidentally go over the maximum token count for some given model source.
This will also be useful in situations where we want to extract the maximum possible response from a model via request_token_num = model_max_token_len - context_tokens_len
Implementation
- The tokenizer should be appropriate for the respective model
- We should use an open-source (Ideally MIT) tokenizer that we can bundle to not require installing additional dependencies
- Dominant language
- Vim Script
- Stars
- 506
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from dense-analysis/neural
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
dense-analysis/neural#56 · 2 comments ·
-
add gemini AI apiOpenenhancement
Difficulty 5/5 Over a week Newbie friendliness 20/100
dense-analysis/neural#50 ·
-
Add support for ClaudeMay be free again @Angelchev claimed this 860 days ago, and no pull request is open. Openenhancement
dense-analysis/neural#48 · 2 comments · 2 reactions · 1 assignee ·
-
Feat: Enhance Neural Buffer UXMay be free again @Angelchev claimed this 1046 days ago, and no pull request is open. Openenhancement
dense-analysis/neural#44 · 1 assignee ·
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 25/100
dense-analysis/neural#43 ·