huggingface/lighteval

[EVAL] Adding PHARE

Open

#696 opened on Apr 30, 2025

View on GitHub
 (9 comments) (0 reactions) (1 assignee)Python (514 forks)auto 404
good first issuehelp wantednew-task

Repository metrics

Stars
 (2,496 stars)
PR merge metrics
 (PR metrics pending)

Description

Evaluation short description

Bias/hallucination eval

Evaluation metadata

Provide all available

Contributor guide

Research direction
Implement the PHARE evaluation benchmark by adding the dataset from Hugging Face (giskardai/phare), implementing the metric and task configuration in the lighteval framework, and creating an evaluation example.
Tech stack
python
Domain
machine learning
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
Python
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60