huggingface/lighteval

[BUG] Installing lighteval breaks hydra-core

Open

#713 opened on May 9, 2025

View on GitHub
 (1 comment) (1 reaction) (0 assignees)Python (514 forks)auto 404
bughelp wanted

Repository metrics

Stars
 (2,496 stars)
PR merge metrics
 (PR metrics pending)

Description

Describe the bug

Installing this package in same env as hydra-core seems to break hydra-core's upstream package called antlr4. Since I use hydra and want to use lighteval as well, is there a way to reconcile these dependencies?

To Reproduce

pip install hydra-core
pip install lighteval

from latex2sympy2_extended.latex2sympy2 import NormalizationConfig

should give you: File "your/conda/env/lib/python3.11/site-packages/antlr4/atn/ATNDeserializer.py", line 89, in adjust v = ord(c) ^^^^^^ TypeError: ord() expected string of length 1, but int found

Expected behavior

To be able to use lighteval + hydra-core functionality.

Version info

python==3.11.12 lighteval==0.9.2 hydra-core==1.3.2

Contributor guide

Research direction
Identify the specific antlr4 version conflict between lighteval and hydra core. Check the dependency trees with `pip show antlr4 python3 runtime` and `pip show lighteval`. Consider pinning antlr4 python3 runtime to a compatible version or using a virtual environment.
Tech stack
python
Domain
backend
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Pythonpip
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60