[Feedback]: Model during a plan is routinely partially completing tasks as a subagent

Open
#6 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Active
Tech stack
vscode
Domain
ai

Research direction

The report concerns MAI-Code-1-Flash used through the VS Code Copilot surface, with no repository files, tests, or entry points identified. First reproduce the model's behavior using a plan and backend-writing task, then compare its completion claims with the implemented feature; done requires determining whether the model consistently reports partially completed criteria as complete.

Written by the indexing model from the issue text.

Description

Model

MAI-Code-1-Flash

Feedback

I figured the best way to use this model would have been to write a plan with a frontier and use MAI-Code for writing backend.

This seems to have partially succeeded consistently. I have observed the MAI model out right lying saying criteria has been completed but when checked for review the feature is partially implemented.

Copilot surface / environment

VS Code

Dominant language
No language data
Stars
44
Forks
2
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/MAI-Code

All issues in microsoft/MAI-Code

Similar issues

More AI Infra & Agents issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.