zilliztech/GPTCache

[Feature]: Support more configs for openAI models

Open

#237 opened on 2023年4月18日

GitHub で見る
 (3 comments) (0 reactions) (1 assignee)Python (407 forks)batch import
help wanted

Repository metrics

Stars
 (5,857 stars)
PR merge metrics
 (30d に merged PR はありません)

説明

Is your feature request related to a problem? Please describe.

Due to the document of openAI, we missed some major parameters of openAI document, see:

https://platform.openai.com/docs/api-reference/completions/create

  1. max_tokens? just bypass to GPT for now
  2. temperature: there are couple things we can do,
    1. randomly pick answer from returned result if they are all very similar.
    2. edit the answer with another small model:For instance image -> https://huggingface.co/lambdalabs/sd-image-variations-diffusers
  3. n -> if there no enough cached result, we will need to generate from OpenAI anyway.
  4. bestof -> control the topk numbers we want to retrieved from cache

Describe the solution you'd like.

No response

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

コントリビューターガイド