Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

`Generator` lets `openai.APIStatusError` escape → one HTTP 4xx/5xx call aborts the whole `description_llm` run

Open
  • Is something wrong?
#351 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
42/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
machine-learning

Research direction

Read src/autointent/generation/_generator.py around _get_structured_output_openai_async and src/autointent/modules/scoring/_description/llm_encoder.py around _process_utterance_async, then compare the fatal-error policy in typesafe.py. Confirm how aiometer.run_all handles per-utterance failures; done means transient failures are retried or degraded without cancelling the batch, while fatal errors stop it with a typed message.

Written by the indexing model from the issue text.

Description

bug

What happens

Generator._get_structured_output_openai_async catches only (ValidationError, ValueError, LengthFinishReasonError) (_generator.py:239), and LLMDescriptionScorer._process_utterance_async catches only RetriesExceededError (llm_encoder.py:214-224). Any openai.APIStatusError (a 4xx the SDK does not retry, or a 408/429/5xx after its 2 built-in retries) and any openai.APIConnectionError therefore propagates out of aiometer.run_all → run_until_complete → Pipeline.fit: the first failed utterance cancels the whole batch and the whole HPO run.

Observed

Darinochka/AutoIntent-experiments#43 (description_llm on openai/gpt-6-astra via OpenRouter, clinc150): OpenRouter answered HTTP 402 in_flight_budget_exhausted on one call, 358/370 calls into the stage. The process crashed, ≈$17.49 of already-made calls produced no result, and the run could only restart from Pipeline.fit (the completed answers were in StructuredOutputCache, but the stage's budget gate then refused the retry). The crash also left two empty cache directories that made every later Generator(...) raise FileNotFoundError until removed by hand — that part is #335.

Proposed

Same policy TypeSafeDescriptionScorer uses in #350 (_is_fatal):

  • transient (408, 429, 5xx, connection errors): retry with backoff honouring Retry-After beyond the SDK's default, then degrade that utterance to a uniform row (as RetriesExceededError already does) and keep going;
  • fatal (401, 403, 404, 422 — and 402 when it is a hard "no credits" stop; OpenRouter's in_flight_budget_exhausted 402 is arguably transient): raise a typed AutoIntent error with a clear message, before more calls are spent;
  • catch per utterance inside the coroutine handed to aiometer.run_all, so one failure never cancels the others in flight.

Follow-up from #350.

Dominant language
Python
Stars
53
Forks
16
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from deeppavlov/AutoIntent

All issues in deeppavlov/AutoIntent

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.