.NET: [Feature]: Integrate Microsoft.Extensions.AI decision-model inference into agent decision points

Description

I recently opened dotnet/extensions#7764 proposing a provider-neutral abstraction in Microsoft.Extensions.AI for decision-oriented AI models.

The proposed model category is different from chat completion: instead of generating arbitrary text, it evaluates application state against bounded questions and returns structured probabilistic decisions such as:

• binary probability;
• categorical choice/distribution;
• ordered score/distribution.

The motivating implementation is TypeSafe AI's Jev, but the proposed MEAI abstraction is intentionally provider-neutral.

This feature request asks whether Microsoft Agent Framework should consume that future MEAI abstraction at agent decision points where a full generative IChatClient call is unnecessary.

The architectural split I am proposing is:

Microsoft.Extensions.AI
    owns the model/inference abstraction

        ↓

Microsoft Agent Framework
    consumes that abstraction for agent orchestration decisions

MAF should not introduce its own parallel decision-model client abstraction.

This issue is therefore dependent on / exploratory relative to:

dotnet/extensions#7764

---

Motivation

Many decisions inside an agent runtime are not themselves generative tasks.

For example:

Should the loop continue?

Which tools are relevant for this turn?

Which skills are relevant?

Which participant should run next?

Does this result satisfy a bounded criterion?

Today these decisions are typically made using one of:

application code
a generative IChatClient
the main agent model itself

Those remain valid options.

However, if Microsoft.Extensions.AI gains a provider-neutral decision-model abstraction, MAF could also use specialized decision-oriented models for these cases.

Conceptually:

                       ┌────────────────────────┐
                       │   Decision Model       │
                       │   IDecisionClient      │
                       └───────────┬────────────┘
                                   │
          ┌────────────────────────┼────────────────────────┐
          │                        │                        │
          ▼                        ▼                        ▼
   Loop evaluation          Tool selection          Skill selection
          │                        │                        │
          └────────────────────────┼────────────────────────┘
                                   │
                                   ▼
                             Agent execution
                                   │
                                   ▼
                              IChatClient

The decision model is an optional auxiliary inference capability.

It does not replace the IChatClient used for reasoning, tool arguments, generation, or the final answer.

---

Proposed integration areas

I think there are three especially useful integration points.

---

1. Decision-model-backed LoopEvaluator

MAF already has an excellent abstraction for this problem:

LoopEvaluator

and an existing implementation:

AIJudgeLoopEvaluator

AIJudgeLoopEvaluator currently uses a separate IChatClient and structured output to decide whether the original request has been sufficiently addressed.

Conceptually:

original request
+ latest agent result
        ↓
IChatClient judge
        ↓
structured JudgeVerdict
        ↓
LoopEvaluation

A decision-oriented model is a natural alternative for the bounded part of this operation:

original request
+ latest agent result
        ↓
IDecisionClient
        ↓
P(completed)
        ↓
threshold / policy
        ↓
LoopEvaluation

For example, assuming the API proposed in MEAI:

var evaluator = new DecisionLoopEvaluator(
    decisionClient,
    new()
    {
        CompletionThreshold = 0.90
    });

var agent = new HarnessAgent(
    chatClient,
    new HarnessAgentOptions
    {
        LoopEvaluators = [evaluator]
    });

The exact API should obviously follow whatever API is ultimately accepted by MEAI.

Important difference from AIJudgeLoopEvaluator

A decision model may be able to answer:

Is the task complete?
P(true) = 0.94

without being able to generate free-form feedback such as:

"The deployment validation step is still missing."

Therefore I do not think DecisionLoopEvaluator should be forced to reproduce every feature of AIJudgeLoopEvaluator.

A useful initial implementation could simply decide:

continue
or
stop

Optional feedback could remain the responsibility of:

• another evaluator;
• a configured callback;
• the main agent;
• or a generative judge when richer analysis is actually needed.

This keeps the decision model in the role it is designed for.

---

2. Pre-model tool selection / shortlisting

Agents can accumulate large tool catalogs.

For example:

150 authorized tools
        ↓
all schemas added to ChatOptions.Tools
        ↓
IChatClient

For sufficiently large catalogs this increases:

• input context;
• model cost;
• tool-selection difficulty;
• ambiguity among similarly described tools.

A decision model could be used as an optional semantic shortlist stage:

all tools
   ↓
authorization / application policy
   ↓
authorized tools
   ↓
decision-model selection
   ↓
relevant subset
   ↓
ChatOptions.Tools
   ↓
IChatClient

For example:

Authorized tools: 120

Decision inference:

filesystem_read     0.01
filesystem_write    0.00
project_inspect     0.91
graph_validate      0.87
graph_apply         0.14
host_restart        0.01

Selected for this turn:

project_inspect
graph_validate

This could potentially be implemented through the existing AIContextProvider lifecycle, since AIContextProvider already participates before invocation and has access to the current tools.

What I would like guidance on is whether MAF should provide a reusable selection abstraction/provider instead of every application implementing this filtering independently.

For example, conceptually:

AIContextProvider
    ↓
candidate tools
    ↓
selection strategy
    ↓
filtered tools

where an MEAI decision-model adapter could be one implementation.

I am not proposing that tool selection itself must depend on a Decision Model.

The more general framework shape may be:

tool candidate source
        ↓
tool selection policy
        ├── application delegate
        ├── deterministic rules
        └── MEAI decision model

This would preserve composition and make decision inference only one optional strategy.

---

Tool-selection security boundary

A decision model must only be able to reduce an already-authorized candidate set.

It must never turn this:

Authorized tools = A, B, C

into:

Available tools = A, B, C, D

where D was not authorized.

The order should always be:

available tools
      ↓
authorization / application policy
      ↓
authorized candidates
      ↓
semantic decision selection
      ↓
model-visible tools

Decision inference is relevance selection, not authorization.

Likewise, selecting a tool does not bypass MAF tool approval.

Decision Model
     ↓
tool becomes visible to agent
     ↓
agent calls tool
     ↓
existing authorization / approval semantics still apply

---

3. Agent Skill selection

The current .NET AgentSkillsProvider implements progressive disclosure:

advertise skills
    ↓
load_skill
    ↓
read resources
    ↓
run scripts

Today, discovered skill names/descriptions are advertised to the agent.

This is effective for a small or moderate number of skills.

For large Skill catalogs, however, a decision-model-backed selection stage could choose the small subset worth advertising for the current interaction:

trusted discovered skills
          ↓
decision model
          ↓
top relevant skills
          ↓
advertise selected skill metadata
          ↓
IChatClient

For example:

80 trusted skills

request:
"Why did this graph execution fail?"

        ↓

decision inference

execution-diagnostics      0.94
graph-debugging            0.86
provider-configuration     0.19
project-creation           0.01
package-publishing         0.00

        ↓

advertise:
execution-diagnostics
graph-debugging

Again, I do not think AgentSkillsProvider should become inherently dependent on IDecisionClient.

A more reusable design might be an optional selection strategy:

AgentSkillsSource
      ↓
trusted discovered skills
      ↓
optional Skill selection strategy
      ↓
advertised skills

with strategies such as:

all skills
application predicate
custom selector
MEAI decision-model selector

The exact API should follow the current Agent Skills design rather than introducing a parallel Skill system.

---

Batch decision inference could be especially useful

One reason a specialized Decision Model is interesting here is that the MEAI proposal explicitly supports evaluating multiple bounded questions against one shared state.

Tool selection could use either:

Choice over candidates

or:

one Binary question per candidate

in a single batch.

For example:

State:
    current user request
    current agent mode
    relevant interaction context

Questions:
    Is tool A relevant?
    Is tool B relevant?
    Is tool C relevant?
    ...

The result may be:

tool A → 0.92
tool B → 0.03
tool C → 0.74

The application/framework selection policy can then apply:

minimum probability
top K
maximum advertised items
fallback behavior

without asking the generative model to perform the selection itself.

---

Existing extension points that may already be sufficient

I do not think every potential use case requires a new MAF API.

There are several existing extension points that already compose well with a future MEAI Decision Client.

---

Group Chat routing

MAF already exposes:

GroupChatManager.SelectNextAgentAsync(...)

which allows application-owned centralized routing.

As discussed in microsoft/agent-framework#7760, this is the intended extension point when application code rather than an active agent should choose the next participant.

An application could therefore implement:

conversation history
      ↓
IDecisionClient Choice
      ↓
selected participant
      ↓
GroupChatManager.SelectNextAgentAsync

without requiring a new orchestration abstraction.

A small first-party adapter or sample might still be useful, but I do not think a new routing hierarchy is necessary.

---

Handoff should remain unchanged

microsoft/agent-framework#7760 clarified that Handoff is intentionally decentralized:

the active agent chooses a target by calling one of the handoff tools.

I am not proposing to change that semantic.

If centralized decision-model routing is required, Group Chat is already the correct MAF pattern.

---

Dynamic per-run configuration

microsoft/agent-framework#7852 discusses dynamic model/tool policy.

The discussion there confirms that the existing combination of:

AgentRunOptions.AdditionalProperties
AIAgent.CurrentRunContext
AIContextProvider

can already carry request-scoped policy while keeping the AIAgent stable.

A Decision Model integration should reuse that run context rather than introduce another request-context abstraction.

---

Agent evaluation

microsoft/agent-framework#7363 covers the bridge between MEAI IEvaluator and MAF IAgentEvaluator.

That is related but distinct.

The proposed IDecisionClient is:

an inference capability

whereas:

IEvaluator / IAgentEvaluator

represent evaluation operations/frameworks.

An evaluator could consume an IDecisionClient, just as it may consume an IChatClient, but I do not think decision-model support should replace the existing evaluation abstractions.

---

Failure and fallback behavior

Decision models are probabilistic and optional.

Any built-in integration should have explicit failure/fallback semantics.

For example, for Tool or Skill selection:

decision service unavailable
or
unsupported primitive
or
result below configured certainty threshold

a safe compatibility fallback could be:

use the already-authorized candidate set

rather than making tools disappear unexpectedly.

This is "fail open" only with respect to relevance filtering, not security:

authorized candidates
      ↓
decision selector fails
      ↓
authorized candidates remain

It must never introduce unauthorized candidates.

For loop completion, the safe default is different.

If the evaluator cannot determine that the task is complete, it should not prematurely terminate the loop.

Conceptually:

decision unavailable / uncertain
      ↓
continue
or
configured fallback evaluator

subject, of course, to the existing maximum-iteration safety cap.

---

Confidence and thresholds belong to policy

The proposed MEAI Decision Client exposes probabilistic results.

MAF should avoid treating:

P = 0.51

as automatically equivalent to a deterministic decision.

Consumers should be able to configure policies such as:

completion threshold = 0.90

minimum tool relevance = 0.70

top K tools = 5

minimum Skill probability = 0.65

and potentially:

if uncertainty is high
    → keep all candidates

or

if uncertainty is high
    → ask the IChatClient

or

if uncertainty is high
    → use another evaluator

The model supplies a probability distribution.

The framework/application supplies the policy.

---

Decision Models should not become authorization engines

This distinction is especially important for agent frameworks.

A Decision Model can answer questions such as:

Is this tool relevant?

Does this output appear complete?

Which participant best matches this request?

Does this content appear risky?

It must not implicitly answer:

Is this user authorized?

May this tool bypass approval?

May this agent access this resource?

Those remain deterministic application/framework policy.

The correct composition is:

authorization / trust policy
            ↓
bounded candidate set
            ↓
Decision Model
            ↓
semantic selection
            ↓
normal Agent Framework execution
            ↓
existing approval / tool policy

---

Data-boundary considerations

A decision model may be backed by a different service/provider from the primary chat model.

Therefore an integration may introduce a second external AI boundary.

For example, Tool selection could send:

user request
tool names/descriptions
current run context

to a Decision Model provider.

As with AIJudgeLoopEvaluator, this has data-exposure implications.

Any first-party integration should make that explicit and should avoid sending more context than is necessary for the bounded decision.

---

Observability

If MEAI introduces a Decision Client, I would prefer MAF to reuse whatever telemetry that client provides rather than defining another model-call telemetry abstraction.

MAF should add agent correlation around those calls where appropriate:

agent run
  ↓
decision inference
  ↓
selection/evaluation result

Useful agent-level attributes/events may include:

decision purpose:
    loop-evaluation
    tool-selection
    skill-selection

candidate count
selected count
fallback used

without recording sensitive candidate text or decision state by default.

---

Proposed initial scope

If the upstream MEAI abstraction is accepted, I think the smallest useful MAF integration would be:

1. First-class decision-backed LoopEvaluator

Something conceptually similar to:

new DecisionLoopEvaluator(
    decisionClient,
    options);

using a Binary decision and configurable completion threshold.

2. A reusable candidate-selection extension point

A general selection mechanism that can operate before the generative model sees a large candidate set.

Potential consumers:

tools
skills

with a Decision Model adapter as one optional implementation.

The selection abstraction itself should not necessarily live in MEAI and should not require a Decision Model.

3. Agent Skills integration with the selection mechanism

Allow AgentSkillsProvider to advertise only the selected subset of trusted/discovered Skills for a run.

---

Things I would leave out of the initial implementation

I would not initially add:

a new MAF IDecisionClient abstraction
Handoff routing changes
a new agent-routing hierarchy
automatic authorization decisions
automatic approval decisions
decision-model-based memory writes
decision-model-based long-term memory
a replacement for IAgentEvaluator / IEvaluator
a replacement for IChatClient
mandatory Decision Model support in AIAgent

Those either already have extension points or represent separate concerns.

---

Possible future integrations

Once the pattern is validated, other consumers could be explored independently:

Group Chat convenience routing
model routing
RAG/context relevance gating
background-agent selection
response quality/cascading
workflow branch selection
guardrail signals

These do not need to be part of the first change.

---

Questions for maintainers

I would especially appreciate guidance on the following:

1. Assuming dotnet/extensions#7764 results in a provider-neutral Decision Client, does MAF consider this a useful inference capability for agent orchestration?

2. Is LoopEvaluator the best first integration point?

3. For tool and Skill shortlisting, should MAF introduce a generic candidate-selection strategy that can be backed by:

   • application code,
   • deterministic rules,
   • or a Decision Client?

4. Or is AIContextProvider already considered sufficient and should MAF only provide a concrete Decision-Model-based provider/helper?

5. Should AgentSkillsProvider expose an optional per-run Skill selector, or should selection happen outside the provider?

6. Should uncertainty/fallback policy be handled by each integration individually or through a reusable selection policy?

7. Should the Decision Client be supplied explicitly to individual components rather than added to HarnessAgent / AIAgent itself?

8. Would a first-party integration belong in Microsoft.Agents.AI, the Harness package, or a separate integration package depending on where the MEAI abstraction lands?

9. For Group Chat, is the current recommendation to simply call the Decision Client from a custom GroupChatManager, rather than introducing another routing API?

10. Are there other bounded decision points in the current framework where a non-generative model capability would fit more naturally?

---

Expected architecture

The goal is ultimately to enable composition such as:

                    Microsoft.Extensions.AI
                    IDecisionClient
                           │
              ┌────────────┼────────────┐
              │            │            │
              ▼            ▼            ▼
         LoopEvaluator    Tools       Skills
              │            │            │
              └────────────┼────────────┘
                           │
                           ▼
                       AIAgent
                           │
                           ▼
                       IChatClient

rather than:

MAF
 ├── its own Decision Model abstraction
 ├── Jev-specific integration
 └── every agent coupled to a Decision Model

The intent is to preserve the same layering that already exists elsewhere:

Microsoft.Extensions.AI
    = model/provider abstractions

Microsoft Agent Framework
    = agent/runtime/orchestration abstractions

and allow agents to combine generative models for generation/reasoning with decision-oriented models for bounded probabilistic decisions where each is appropriate.

Code Sample

Conceptually, assuming the MEAI API proposed in dotnet/extensions#7764:

IDecisionClient decisionClient = ...;
IChatClient chatClient = ...;

var completionEvaluator = new DecisionLoopEvaluator(
    decisionClient,
    new DecisionLoopEvaluatorOptions
    {
        CompletionThreshold = 0.90
    });

var agent = new HarnessAgent(
    chatClient,
    new HarnessAgentOptions
    {
        LoopEvaluators = [completionEvaluator],

        AIContextProviders =
        [
            new DecisionToolSelectionProvider(
                decisionClient,
                new()
                {
                    MaximumSelectedTools = 8,
                    MinimumProbability = 0.60
                })
        ]
    });

And for Skills, conceptually:

var skillsProvider = new AgentSkillsProvider(
    skillsSource,
    new AgentSkillsProviderOptions
    {
        SkillSelector = new DecisionSkillSelector(
            decisionClient,
            maximumSelectedSkills: 5)
    });

The exact public APIs are intentionally not prescribed here because they should follow the final MEAI abstraction and existing MAF design patterns.

The important architectural requirement is that MAF consumes the MEAI decision-model capability rather than defining a second provider abstraction.

Language/SDK

.NET