huggingface/lighteval

[EVAL] Add kyrgyzLLM benchmark

Aperta

#1036 aperta il 4 nov 2025

 (1 commento) (0 reazioni) (1 assegnatario)Python (514 fork)auto 404
good first issuenew-task

Metriche repository

Star
 (2496 stelle)
Metriche merge PR
 (Metriche PR in attesa)

Descrizione

Hi,

We just open-sourced the Kyrgyz LLM Evaluation Dataset.

Evaluation short description

  • Why is this evaluation interesting?

KyrgyzLLM-Bench is the first comprehensive benchmark suite for deep language understanding in Kyrgyz. It is interesting because it provides broad, culturally grounded coverage by combining native benchmarks (such as KyrgyzMMLU and KyrgyzRC) with carefully translated and post-edited international benchmarks (such as HellaSwag, WinoGrande, BoolQ, GSM8K, and TruthfulQA).

  • How used is it in the community? As the benchmark was released recently, its adoption by the community is just beginning. It is significant because it's the first comprehensive benchmark suite for deep language understanding, specifically in the Kyrgyz language, providing a new and essential tool for researchers and developers.

Evaluation metadata

Thanks you!

Guida contributor