Sample proposal: EvalPort ↔ azure-ai-evaluation interchange under scenarios/evaluate

Start with scenarios/evaluate/README.md, the Supported_Evaluation_Targets/ and Supported_Evaluation_Metrics/ examples, and CONTRIBUTING.md for the notebook template. Use the existing azure-ai-evaluation adapter as the reference for the sample, and consider the work complete when a self-contained notebook demonstrates exporting a suite and result set and validates both.

Hi — I maintain EvalPort (Apache 2.0), an open JSON spec for portable LLM evaluation datasets: test suites, test cases, graders, and result sets that can move between eval frameworks without losing meaning.

I looked through scenarios/evaluate/ before opening this — the Supported_Evaluation_Targets/ and Supported_Evaluation_Metrics/ structure (and the comparison table in scenarios/evaluate/README.md) is a genuinely useful map of what azure-ai-evaluation supports, which is why I think this is worth a shot.

There's already a working, tested adapter for azure-ai-evaluation in EvalPort's adapters/ directory: azure-ai-evaluation-openeval-adapter. It's a small package with to_openeval() / from_openeval() / evaluation_result_to_openeval(), tested against the real azure-ai-evaluation package (21 tests, not mocks) — not a proposal, an existing reference implementation.

A minimal version of what a sample notebook would show:

from azure.ai.evaluation import F1ScoreEvaluator
from azure_ai_evaluation_openeval_adapter import to_openeval, evaluation_result_to_openeval
from openeval.validate import validate_suite, validate_result_set

# Your evaluate()-shaped data + evaluators, exported as a portable EvalPort suite
suite = to_openeval(
    data="my_eval_data.jsonl",
    evaluators={"f1": F1ScoreEvaluator()},
    suite_id="my_eval_suite",
)
assert validate_suite(suite).valid

# ...run azure.ai.evaluation.evaluate() as usual, then export the result:
result_set = evaluation_result_to_openeval(result, suite_id="my_eval_suite")
assert validate_result_set(result_set).valid

Would a small notebook like this be a welcome addition under scenarios/evaluate/ (e.g. alongside Supported_Evaluation_Targets/)? It'd be scoped as one self-contained sample per your CONTRIBUTING.md guidance — happy to follow the README/notebook template if there's interest. No worries at all if this isn't a fit for the repo's current scope — just flagging since the adapter already exists and works.

Spec: https://github.com/adhabnr-ux/evalport/blob/main/spec/SPEC.md