Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

增加对GLM-4大模型的支持

Aperta
  • Qualcosa non va?
#382 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
45/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Ferma
Stack tecnologico
python
Ambito
ai, backend

Direzione di ricerca

Inizia in camel/model_backend.py, dove viene chiamato tiktoken.encoding_for_model, e riproduci il comando riportato con GLM_4 e la BASE_URL fornita. Traccia come vengono selezionati i nomi dei modelli e le codifiche dei token. Il lavoro è completato quando GLM-4 può essere invocato tramite questo comando senza il KeyError del tokenizer segnalato.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

调用方法：
export BASE_URL="https://open.bigmodel.cn/api/paas/v4/"
export OPENAI_API_KEY="XXXXXXXXXXXXXXXXXXXX"
python run.py --task "[Develop a snake game]" --name "[snamer03]" --model=GLM_4

错误日志：
...
Note that we must ONLY discuss the product modality and do not discuss anything else! Once we all have expressed our opinion(s) and agree with the results of the discussion unanimously, any of us must actively terminate the discussion by replying with only one line, which starts with a single word , followed by our final product modality without any other words, e.g., " PowerPoint".

Traceback (most recent call last):
File "/Users/rui/anaconda3/envs/ChatDev_conda_env/lib/python3.9/site-packages/tenacity/init.py", line 382, in call
result = fn(*args, **kwargs)
File "/Users/rui/workbrench/ChatDev/camel/utils.py", line 157, in wrapper
return func(self, *args, **kwargs)
File "/Users/rui/workbrench/ChatDev/camel/agents/chat_agent.py", line 239, in step
response = self.model_backend.run(messages=openai_messages)
File "/Users/rui/workbrench/ChatDev/camel/model_backend.py", line 68, in run
encoding = tiktoken.encoding_for_model(self.model_type.value)
File "/Users/rui/anaconda3/envs/ChatDev_conda_env/lib/python3.9/site-packages/tiktoken/model.py", line 70, in encoding_for_model
raise KeyError(
KeyError: 'Could not automatically map glm-4 to a tokeniser. Please use tiktok.get_encoding to explicitly get the tokeniser you expect.'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/rui/workbrench/ChatDev/run.py", line 135, in
chat_chain.execute_chain()
File "/Users/rui/workbrench/ChatDev/chatdev/chat_chain.py", line 168, in execute_chain
self.execute_step(phase_item)
File "/Users/rui/workbrench/ChatDev/chatdev/chat_chain.py", line 138, in execute_step
self.chat_env = self.phases[phase].execute(self.chat_env,
File "/Users/rui/workbrench/ChatDev/chatdev/phase.py", line 295, in execute
self.chatting(chat_env=chat_env,
File "/Users/rui/workbrench/ChatDev/chatdev/utils.py", line 79, in wrapper
return func(*args, **kwargs)
File "/Users/rui/workbrench/ChatDev/chatdev/phase.py", line 133, in chatting
assistant_response, user_response = role_play_session.step(input_user_msg, chat_turn_limit == 1)
File "/Users/rui/workbrench/ChatDev/camel/agents/role_playing.py", line 247, in step
assistant_response = self.assistant_agent.step(user_msg_rst)
File "/Users/rui/anaconda3/envs/ChatDev_conda_env/lib/python3.9/site-packages/tenacity/init.py", line 289, in wrapped_f
return self(f, *args, **kw)
File "/Users/rui/anaconda3/envs/ChatDev_conda_env/lib/python3.9/site-packages/tenacity/init.py", line 379, in call
do = self.iter(retry_state=retry_state)
File "/Users/rui/anaconda3/envs/ChatDev_conda_env/lib/python3.9/site-packages/tenacity/init.py", line 326, in iter
raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x119747fd0 state=finished raised KeyError>]

代码定位：
/site-packages/tiktoken/model.py
MODEL_TO_ENCODING: dict[str, str] = {
# chat
"gpt-4": "cl100k_base",
"gpt-3.5-turbo": "cl100k_base",
# text
"text-davinci-003": "p50k_base",
"text-davinci-002": "p50k_base",
"text-davinci-001": "r50k_base",
"text-curie-001": "r50k_base",
"text-babbage-001": "r50k_base",
"text-ada-001": "r50k_base",
"davinci": "r50k_base",
"curie": "r50k_base",
"babbage": "r50k_base",
"ada": "r50k_base",
# code
"code-davinci-002": "p50k_base",
"code-davinci-001": "p50k_base",
"code-cushman-002": "p50k_base",
"code-cushman-001": "p50k_base",
"davinci-codex": "p50k_base",
"cushman-codex": "p50k_base",
# edit
"text-davinci-edit-001": "p50k_edit",
"code-davinci-edit-001": "p50k_edit",
# embeddings
"text-embedding-ada-002": "cl100k_base",
# old embeddings
"text-similarity-davinci-001": "r50k_base",
"text-similarity-curie-001": "r50k_base",
"text-similarity-babbage-001": "r50k_base",
"text-similarity-ada-001": "r50k_base",
"text-search-davinci-doc-001": "r50k_base",
"text-search-curie-doc-001": "r50k_base",
"text-search-babbage-doc-001": "r50k_base",
"text-search-ada-doc-001": "r50k_base",
"code-search-babbage-code-001": "r50k_base",
"code-search-ada-code-001": "r50k_base",
# open source
"gpt2": "gpt2",
}

def encoding_for_model(model_name: str) -> Encoding:
"""Returns the encoding used by a model."""
encoding_name = None
if model_name in MODEL_TO_ENCODING:
encoding_name = MODEL_TO_ENCODING[model_name]
else:
# Check if the model matches a known prefix
# Prefix matching avoids needing library updates for every model version release
# Note that this can match on non-existent models (e.g., gpt-3.5-turbo-FAKE)
for model_prefix, model_encoding_name in MODEL_PREFIX_TO_ENCODING.items():
if model_name.startswith(model_prefix):
return get_encoding(model_encoding_name)

if encoding_name is None:
    raise KeyError(
        f"Could not automatically map {model_name} to a tokeniser. "
        "Please use `tiktok.get_encoding` to explicitly get the tokeniser you expect."
    ) from None

return get_encoding(encoding_name)
Lingua principale
Python
Stelle
34.5k
Fork
4.3k
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

  • Include un Dockerfile o un file Docker Compose
  • Nessun modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di OpenBMB/ChatDev

Tutte le issue di OpenBMB/ChatDev

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.