Term Cleanup: Handle embedding generation errors like rate limits
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
Research direction
The issue does not name files or tests; first locate the cleanup flow that generates embeddings and inspect how API responses and rate-limit failures are currently handled. Done means general API errors stop with a visible message, while rate-limit responses retry with bounded backoff or eventually stop with an error, backed by tests for both paths.
Written by the indexing model from the issue text.
Description
Is your enhancement related to a problem? Please describe.
Originally documented by @dkotter:
I didn't run into any errors when testing but in looking at the code, it doesn't seem we ever look at the response from generating an embedding.
So in a scenario where this API request fails, we just continue on to the next term. Think of a situation where the API goes down or we hit rate limits on the API. In that case, the process will continue to try forever even though we aren't getting any results from the API.
I'd suggest, if easy enough, two different approaches:
- If we get just a general error response from the API (i.e. the API is down), I think we should show that error message and stop the process from continuing
- If we get a rate limit error, ideally we would pause for a minute or two before proceeding and then try again. If it fails again with a rate limit error, maybe increase our pause by a couple minutes. Once we've tried this a few times, we can then output an error and stop the process. So basically an automatic retry mechanism with a built-in backoff mechanic to try and handle rate limit issues.
Note this last piece may be overkill for this so we can discuss whether it's worth the effort or not.
Designs
Originally documented by @dkotter:
Note that OpenAI exposes some headers around rate limits that would be interesting to use: https://platform.openai.com/docs/guides/rate-limits/rate-limits-in-headers
They also provide some examples (in python) on how to build an exponential backoff system: https://platform.openai.com/docs/guides/rate-limits/retrying-with-exponential-backoff
Describe alternatives you've considered
Originally reported via https://github.com/10up/classifai-cleanup/issues/3
Code of Conduct
- I agree to follow this project's Code of Conduct
- Dominant language
- PHP
- Stars
- 714
- Forks
- 68
- Avg merge
- 4h 32m
- Merged PRs (30d)
- 23
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from 10up/classifai
-
wpvc
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
wpvc
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
Register ClassifAI functionality with the Abilities APIPossibly taken @nhrrob claimed this 10 days ago. Openhelp wanted
Difficulty 4/5 3-5 days Newbie friendliness 48/100
10up/classifai#1249 · 1 assignee ·
Maintainers usually reply within 1 day
-
OpenAI usage tracking: Project ID setting is ignoredPossibly taken @dajiaohuang claimed this 30 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
BYOK Openai modelsOpen
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
Similar issues
-
Exclude module `require-dev` and `autoload-dev` from the generated `composer.json`Possibly taken @keiserjb claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
uiowa/uiowa#10310 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
mailcow/mailcow-dockerized#7677 · 1 comment ·
Maintainers usually reply within 1 day
-
a11y
Difficulty 2/5 1-3 hours Newbie friendliness 67/100