Gilfeather/furnace

⚡ Add request timeout handling

開放

#10 建立於 2025年7月16日

 (0 則留言) (0 個反應) (0 位負責人)Rust (5 個分叉)auto 404
good first issuehelp wanted

倉庫指標

星標
 (64 顆星)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

Description

Implement request timeout handling to prevent long-running requests from blocking the server.

Current Status - Partially Implemented

Model-level timeout exists but HTTP-level timeout handling is not implemented.

Already Implemented

  • Model-level timeout in predict_batch_with_timeout method
  • Timeout logging and monitoring
  • Graceful timeout error handling at model level

Remaining Work

  • Add configurable HTTP request timeout (default 30 seconds)
  • Return HTTP 408 Request Timeout for exceeded requests
  • Add timeout configuration to CLI arguments
  • Add timeout middleware to Axum router
  • Add tests for HTTP timeout behavior
  • Update documentation with timeout information

Implementation Guidance

  • Use tower::timeout::TimeoutLayer for HTTP-level timeouts
  • Add --request-timeout CLI argument
  • Consider different timeouts for different endpoints
  • Integrate with existing model timeout for layered timeout handling

Estimated Difficulty

Easy - Half day (reduced due to existing timeout infrastructure)

貢獻者指南