Gilfeather/furnace

⚡ Add request timeout handling

Offen

#10 geöffnet am 16.07.2025

 (0 Kommentare) (0 Reaktionen) (0 zugewiesene Personen)Rust (5 Forks)auto 404
good first issuehelp wanted

Repository-Metriken

Stars
 (64 Sterne)
PR-Merge-Metriken
 (Keine gemergten PRs in 30 T)

Beschreibung

Description

Implement request timeout handling to prevent long-running requests from blocking the server.

Current Status - Partially Implemented

Model-level timeout exists but HTTP-level timeout handling is not implemented.

Already Implemented

  • Model-level timeout in predict_batch_with_timeout method
  • Timeout logging and monitoring
  • Graceful timeout error handling at model level

Remaining Work

  • Add configurable HTTP request timeout (default 30 seconds)
  • Return HTTP 408 Request Timeout for exceeded requests
  • Add timeout configuration to CLI arguments
  • Add timeout middleware to Axum router
  • Add tests for HTTP timeout behavior
  • Update documentation with timeout information

Implementation Guidance

  • Use tower::timeout::TimeoutLayer for HTTP-level timeouts
  • Add --request-timeout CLI argument
  • Consider different timeouts for different endpoints
  • Integrate with existing model timeout for layered timeout handling

Estimated Difficulty

Easy - Half day (reduced due to existing timeout infrastructure)

Contributor Guide