Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

About appropriate groups_per_step parameter setting given training and validation dataset volume

Aperta
  • Qualcosa non va?
#397 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 3 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
20/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Ferma
Stack tecnologico
python
Ambito
backend, machine-learning

Direzione di ricerca

Iniziare con dev/art-e/train.py e la gestione di groups_per_step, quindi riprodurre i casi 100/50 e 300/100 esaminando al contempo i grafici del monitor vLLM e i log dei timeout allegati. Il lavoro è completato quando viene identificato se l’impostazione del parallelismo causa i timeout del rollout e viene documentata un’impostazione supportata o una mitigazione.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

question

We basically implemented our trainer based on https://github.com/OpenPipe/ART/blob/5a60fa017ab876910bbec61add43f81ef4103eb9/dev/art-e/art_e/train.py. Essentially, the same code skeleton for local vllm hosting.

However, during the experiment, one werid issue raised when setting groups_per_step = 50/40:

  1. On a small training and validation dataset, for instance, 100, 50, the end-to-end training can be completed successfully. (groups_per_step=50)
  2. Scaling out the training and validation daset to 300, 100, then end-to-end training can't be completed because LLM client reports a lot of timeout issue during rollout. (both groups_per_step = 50/40). The exception traces as following:
2025-09-08 10:56:55.806 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:56:55.845 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:56:56.022 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:56:56.416 INFO _base_client - _sleep_for_retry: Retrying request to /chat/completions in 0.902451 seconds
2025-09-08 10:56:56.422 INFO _base_client - _sleep_for_retry: Retrying request to /chat/completions in 0.842315 seconds
2025-09-08 10:56:56.460 ERROR rl_train_func - rollout_per_prediction_turn: API call error (attempt 1/3): litellm.InternalServerError: InternalServerError: Hosted_vllmException - Connection error.
2025-09-08 10:56:56.460 INFO rl_train_func - rollout_per_prediction_turn: Waiting for 0.22 seconds before retrying...
2025-09-08 10:56:57.790 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:56:57.924 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:56:58.015 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:56:58.555 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:56:58.757 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:57:00.028 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:57:00.160 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:57:04.125 INFO _base_client - _sleep_for_retry: Retrying request to /chat/completions in 0.804868 seconds
2025-09-08 10:57:04.486 ERROR llm_utils - parse_cot_result: Error parsing COT result: No <think> section found in the response
2025-09-08 10:57:05.256 ERROR rl_train_func - rollout_per_prediction_turn: API call error (attempt 1/3): litellm.Timeout: APITimeoutError - Request timed out. Error_str: Request timed out. - timeout value=600.0, time taken=1813.92 seconds
2025-09-08 10:57:05.256 INFO rl_train_func - rollout_per_prediction_turn: Waiting for 0.91 seconds before retrying...
2025-09-08 10:57:05.263 ERROR rl_train_func - rollout_per_prediction_turn: API call error (attempt 1/3): litellm.Timeout: APITimeoutError - Request timed out. Error_str: Request timed out. - timeout value=600.0, time taken=1814.36 seconds
2025-09-08 10:57:05.264 INFO rl_train_func - rollout_per_prediction_turn: Waiting for 0.44 seconds before retrying...
2025-09-08 10:57:05.271 ERROR rl_train_func - rollout_per_prediction_turn: API call error (attempt 1/3): litellm.Timeout: APITimeoutError - Request timed out. Error_str: Request timed out. - timeout value=600.0, time taken=1813.89 seconds

I compared with the vllm monitor for the successful training (pink line), the failed training (green line).

Image Image Image

It seems virtual memory allocation of the vllm hosting process looks abnormal.

Another information about data volume is:
Each data point is actually a dialogue conversation with at most 20 turns. We let LLM expand for specific turns in this conversation, each of them will rollout 4 times.

For my understanding, in art groups_per_step decides how many parallel requests are sent to the local hosted VLLM. Considering no optimized vllm setting, I can understand the delayed client requests will be queued until they are consumed by VLLM. I observed the vllm monitor diagrams and found the corresponding time delays.
Maybe the accumulated delayed requests can't be process timely by VLLM so that LLM clients encountered the issue of timeout. But why the virtual memory of hosting processes keeps at a high level until the VLLM client reports timeout?

Can we have some ways to solve/alleviate this issue? We run our experiments on a powerful machine, so maybe an appropriate setting can help us address this issue.

Best Regards
Orlando

Lingua principale
Python
Stelle
10.8k
Fork
989
Merge medio
11h 38m
PR unite (30g)
104

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di OpenPipe/ART

Tutte le issue di OpenPipe/ART

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.