microsoft/autogen
Vedi su GitHubAuto Recovery Logic for chat completion clients on different types of server errors
Open
#3632 aperta il 3 ott 2024
help wantedproj-extensionssize-medium
Metriche repository
- Star
- (58.033 star)
- Metriche merge PR
- (Nessuna PR mergiata in 30 g)
Descrizione
What feature would you like to be added?
It would be nice instead if we can support a model client that auto-recovers from various host-related errors with a configurable logic such as retries.
Here is a good example of an apparently transient error:
openai.APIStatusError: Error code: 424 - {'error': {'message': 'Error occurred while processing image(s).', 'type': 'failed_dependency', 'param': None, 'code': None}}
Moreover, even for rate limit errors like below, it doesn't always retry, there is some inconsistency happening
openai.RateLimitError: Error code: 429 - {'error': {'code': '429', 'message': 'Rate limit is exceeded. Try again in 1 seconds.'}}
Why is this needed?
Currently, the implementation of the chat completion client for OpenAI fails to recover and crashes for a host of errors (except for retries that are rate-limit related where it auto-retries), however, a host of other errors randomly occur and that forces the applications built on top of the client to have their own logic for auto-recovery.