zilliztech/GPTCache

[Feature]: Support more configs for openAI models

Open

#237 创建于 2023年4月18日

在 GitHub 查看
 (3 评论) (0 反应) (1 负责人)Python (407 fork)batch import
help wanted

仓库指标

Star
 (5,857 star)
PR 合并指标
 (30 天内没有已合并 PR)

描述

Is your feature request related to a problem? Please describe.

Due to the document of openAI, we missed some major parameters of openAI document, see:

https://platform.openai.com/docs/api-reference/completions/create

  1. max_tokens? just bypass to GPT for now
  2. temperature: there are couple things we can do,
    1. randomly pick answer from returned result if they are all very similar.
    2. edit the answer with another small model:For instance image -> https://huggingface.co/lambdalabs/sd-image-variations-diffusers
  3. n -> if there no enough cached result, we will need to generate from OpenAI anyway.
  4. bestof -> control the topk numbers we want to retrieved from cache

Describe the solution you'd like.

No response

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

贡献者指南