Add scheduled end-to-end validation for Go examples

Open
#925 0 comments 0 reactions 1 assignee View on GitHub

@michelle-clayton-work is already working on this.

Since Aug 27, 2026.

Assessment

This issue has not been assessed yet.

Description

Problem

cmd/verifyexamples runs the Go examples and checks their output, but no CI workflow currently runs it against live model providers.

Unit tests use mocked providers, so they do not catch authentication failures, provider API changes, or other problems that only appear in a real model call.

PR #431 added verifyexamples and left scheduled CI as follow-up work once provider access was available.

Proposed work

Add a scheduled and manually dispatched GitHub Actions workflow that:

  • Runs a small, stable set of examples through cmd/verifyexamples.
  • Writes the Markdown report to the GitHub job summary.
  • Uploads the detailed report as an artifact.
  • Uses short-lived authentication where possible.
  • Keeps API keys and tokens out of the repository.
  • Starts as an informational check rather than a required PR check.

Provider investigation

Start by checking whether the GitHub Copilot provider can use authentication already available to GitHub Actions. Copilot-only coverage would still test real agent calls without requiring accounts for every supported provider.

verifyexamples currently uses Foundry for semantic output verification. If Copilot is the first provider tested, decide whether to:

  • Use deterministic output checks.
  • Add Copilot-backed semantic verification.
  • Keep Foundry as the verifier and configure Azure OIDC.

For broader provider coverage, ask the Agent Framework team how they provision and manage credentials for their .NET and Python integration workflows. Anthropic and Gemini may require separate paid credentials.

Done when

  • A scheduled and manually dispatched workflow runs selected examples.
  • At least one live provider path runs end to end.
  • Missing credentials fail or skip with a clear reason.
  • The workflow publishes a report even when an example fails.
  • Authentication and ownership are documented.
  • No reusable credential is committed to the repository.

References

Dominant language
Go
Stars
619
Forks
56
Avg merge
1d 2h
Merged PRs (30d)
131

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/agent-framework-go

All issues in microsoft/agent-framework-go

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.