neulab/prompt2model

tiktoken usage

Open

#341 opened on 2023年9月8日

GitHub で見る
 (2 comments) (0 reactions) (0 assignees)Python (137 forks)batch import
good first issue

Repository metrics

Stars
 (1,663 stars)
PR merge metrics
 (30d に merged PR はありません)

説明

https://github.com/neulab/prompt2model/pull/335#issuecomment-1711578817

Another thing is that I am not sure that all the API-based models use tiktoken to compute input tokens. Thus, the num of input tokens may slightly differ between these models. But that's minimal.

コントリビューターガイド