Q57: live MeasuringWorth reference differs from fixed answer key

Abierto
  • ¿Algo no está bien?
#58 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
32/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
jupyter-notebook
Área
data, documentation

Línea de trabajo

Start from officeqa_pro_v2.csv, question ID 57's source_docs metadata and the MeasuringWorth comparator URL it cites; compare the live Relative Labor Earnings output against the stored expected answer. Read databricks/officeqa#47 for the related convention debate. Done means the metadata pins a dated reference snapshot or an explicit conversion factor so the key reproduces within 1%, plus a reply to the reporter requesting the private reproduction.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Dataset and question

Dataset: databricks/officeqa-pro-v2
File: officeqa_pro_v2.csv
Question ID: 57

Reproducibility discrepancy

On October 6, 2026, I checked the external MeasuringWorth compensation comparator referenced in this question’s source_docs metadata.

Using the nominal amount from the cited Treasury document, the live calculator’s Relative Labor Earnings result differs from the dataset’s expected answer by approximately 2.06%.

The calculator labels its output “today,” while the question specifies 2026 dollars. The reference URL does not pin a target year or source vintage.

I have not established whether this reflects a data revision, a different reference vintage, or ambiguity in the target-year interpretation. However, the current reference does not reproduce the fixed answer key, even within a 1% tolerance.

Requested clarification

Could you clarify:

  • Which source vintage and conversion factor were used to generate the answer?
  • How “2026 dollars” should be interpreted for this comparator?
  • Whether a dated reference snapshot or explicit conversion factor can be provided to make the question reproducible?

This appears related to the answer-revision and convention-clarification concerns raised in databricks/officeqa#47, although this report concerns Pro V2.

Following the approach in that issue, I am withholding the full question, answer values, and detailed calculation from this public report to avoid benchmark contamination. I can share the exact reproduction privately with the maintainers.

Lenguaje dominante
Jupyter Notebook
Estrellas
205
Forks
17
Sin PR fusionados en 30 d

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de databricks/officeqa

Todos los issues de databricks/officeqa

Issues similares

Más issues de Data Engineering

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.