I got the error on centos 7: failed to tokenize string!
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 30/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- cpp
- Domain
- machine-learning
Research direction
Start by reproducing the reported command on CentOS 7 with chinese-alpaca-7b-q4.bin and two threads. Investigate the tokenizer failure shown after entering chat mode; done means identifying the cause and confirming that the same invocation no longer exits with the reported error and Killed message.
Written by the indexing model from the issue text.
Description
I got the error like this on centos 7:
failed to tokenize string!
[root@VM-0-15-centos build]# ./llama -m chinese-alpaca-7b-q4.bin -t 2
main: seed = 1700961097
model is new , version = 1
load: n_vocab = 49954
load: n_ctx = 2048
load: n_embd = 4096
load: n_mult = 256
load: n_head = 32
load: n_layer = 32
load: n_rot = 128
load: model ftype = 2
total weight length = 4304332800
main: interactive mode on.
sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000
== Running in chat mode. ==
- Press Ctrl+C to interject at any time.
- If you want to submit another line, end your input in ''.
failed to tokenize string!
Killed
Originally posted by @Mignet in https://github.com/MegEngine/InferLLM/issues/28#issuecomment-1826459644
- Dominant language
- C++
- Stars
- 752
- Forks
- 94
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from MegEngine/InferLLM
-
SG2042 服务器 make 出错 Open
Difficulty 4/5 3-5 days Newbie friendliness 30/100
-
无法加载模型bin或者guff模型 Open
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
有打算支持qwen吗 Open
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 4/5 3-5 days Newbie friendliness 20/100
-
这个有windows的部署教程吗 Open
Difficulty 3/5 1-2 days Newbie friendliness 35/100
All issues in MegEngine/InferLLM
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
ros2/message_filters#338 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
subsurface/subsurface#4984 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
flutter-webrtc/flutter-webrtc#2206 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
google-ai-edge/LiteRT-LM#3739 ·