Auto Recovery Logic for chat completion clients on different types of server errors · microsoft/autogen#3632

(4 留言) (0 反應) (0 負責人)Python (58,033 star) (8,759 fork)batch import

help wantedproj-extensionssize-medium

描述

What feature would you like to be added?

It would be nice instead if we can support a model client that auto-recovers from various host-related errors with a configurable logic such as retries.

Here is a good example of an apparently transient error:

openai.APIStatusError: Error code: 424 - {'error': {'message': 'Error occurred while processing image(s).', 'type': 'failed_dependency', 'param': None, 'code': None}}

Moreover, even for rate limit errors like below, it doesn't always retry, there is some inconsistency happening

openai.RateLimitError: Error code: 429 - {'error': {'code': '429', 'message': 'Rate limit is exceeded. Try again in 1 seconds.'}}

Why is this needed?

Currently, the implementation of the chat completion client for OpenAI fails to recover and crashes for a host of errors (except for retries that are rate-limit related where it auto-retries), however, a host of other errors randomly occur and that forces the applications built on top of the client to have their own logic for auto-recovery.

貢獻者指南

技術棧: python
領域: backend
議題類型: feature
難度: 3
預計時間: 1-3 hours
活動狀態: fresh
清晰度: clear
前置要求: PythonOpenAI APIRetry strategiesAutoGen contribution guide
新手友善度: 65
研究方向: Examine the existing client code in autogen/oai/client.py or similar to understand current retry logic for rate limits. Implement a retry mechanism that can be configured via parameters to handle 424 and 429 errors. Use a library like tenacity for configurable retries. Ensure the solution aligns with the existing error handling pattern.