Release ToolSense benchmarks on Hugging Face
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Aptitud para principiantes
- 45/100
Línea de trabajo
Start by reviewing the pre-generated JSONL files in the data/ folder and the Hugging Face datasets loading guide. Determine how the RRB, MCQ Probing, and QA Probing benchmarks should be uploaded as datasets and linked to the paper page. Done means the datasets are hosted on Hugging Face and discoverable from the paper.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Hi @ashutosh1919 🤗
I'm Niels and work as part of the open-source team at Hugging Face. I discovered your work on Arxiv and was wondering whether you would like to submit it to hf.co/papers to improve its discoverability. If you are one of the authors, you can claim the paper at https://huggingface.co/papers/2606.12451.
The paper page lets people discuss about your paper and lets them find artifacts about it (your benchmarks for instance), you can also claim the paper as yours which will show up on your public profile at HF, and add Github and project page URLs.
Would you like to host the datasets you've released (the RRB, MCQ Probing, and QA Probing benchmarks) on https://huggingface.co/datasets?
I see you're currently hosting the pre-generated JSONL files in the data/ folder of your GitHub repository. Hosting on Hugging Face will give you more visibility/enable better discoverability, and will also allow people to do:
from datasets import load_dataset
# Example for the retrieval benchmark
dataset = load_dataset("your-hf-org-or-username/toolsense-rrb")
If you're down, leaving a guide here: https://huggingface.co/docs/datasets/loading.
Besides that, there's the dataset viewer which allows people to quickly explore the first few rows of the data directly in the browser.
After uploaded, we can also link the datasets to the paper page (read here) so people can discover your work.
Let me know if you're interested/need any guidance.
Kind regards,
Niels
- Lenguaje dominante
- Python
- Estrellas
- 1
- Forks
- 3
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de SAP/toolsense
-
Add a model-agnostic evaluation harness for the QA/MCQ/RRB benchmarksPosiblemente ocupada @Gazzara-lab la tomó hace 132 días. Abierto
Dificultad 4/5 3-5 días Aptitud para principiantes 55/100
Todos los issues de SAP/toolsense
Issues similares
-
[oblt-aw][autodoc] Update docs for security org scaffolding and Docs AI menu permission changesPosiblemente ocupada @github-actions la tomó hoy. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
Los mantenedores suelen responder en 1 día
-
external integration integration-submission
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
langchain-ai/docs#7077 ·
Los mantenedores suelen responder en 1 día
-
author-page correction
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
acl-org/acl-anthology#10623 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Use context to recognize personal initials and lowercase namesPosiblemente ocupada @lipikaramaswamy la tomó hoy. Abiertobug
Dificultad 2/5 1-3 horas Aptitud para principiantes 67/100
NVIDIA-NeMo/Anonymizer#340 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 82/100
btclib-org/btclib-node#2083 ·
Los mantenedores suelen responder en 1 día