jxnl/instructor-classify

Add cross-provider cost tracking support

Aperta

#2 aperta il 19 apr 2025

 (0 commenti) (0 reazioni) (0 assegnatari)Python (4 fork)auto 404
enhancementgood first issue

Metriche repository

Star
 (37 stelle)
Metriche merge PR
 (Metriche PR in attesa)

Descrizione

Cost tracking currently only works for OpenAI and does not use the instructor usage tracking standard.

Description

Extend the cost tracking functionality to work with all providers supported by Instructor, not just OpenAI. This involves integrating with Instructor's standardized usage tracking system.

Tasks

  • Review Instructor's usage tracking standard implementation
  • Update cost analysis module to work with this standard format
  • Add provider-specific cost calculations for Anthropic, Google, etc.
  • Add tests for each provider's cost tracking
  • Update documentation to reflect cross-provider cost tracking

Expected Result

Users should be able to see accurate cost tracking regardless of which LLM provider they use. The evaluation framework should produce cost metrics for all supported providers.

Difficulty

Medium - requires understanding of Instructor's usage tracking system and multiple provider APIs

Guida contributor