[Bug]: Incorrect AI percentage calculation for mixed commits (human + AI changes)

Open Beginner friendly
#1,425 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
74/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
git, rust
Domain
devtools

Research direction

Start in src/authorship/stats.rs at accepted_lines_from_attestations(), especially lines 619-629, and reproduce the behavior with git-ai stats HEAD using an invalid attestation hash such as ai_agent. Ensure hashes absent from metadata.prompts are not counted as AI lines and are instead categorized as unknown additions, while valid prompt hashes still produce the expected authorship percentages.

Written by the indexing model from the issue text.

Description

bug
What broke?

Description

When a commit contains both human-authored and AI-generated changes, git-ai incorrectly calculates the AI authorship percentage. Lines that cannot be matched to valid prompt records are incorrectly counted as AI-generated instead of being treated as human/unknown.

Why This is a Problem

In our test case, the git notes contained:

src/array.ts
  ai_agent 2
  • "ai_agent" is a CheckpointKind marker, not a valid prompt session hash
  • The entry is parsed as a valid attestation with hash="ai_agent"
  • But "ai_agent" doesn't exist in metadata.prompts
  • Despite this, the code counts these 2 lines as AI modifications

Environment

  • git-ai version: 1.4.11
  • OS: macOS (Darwin 24.6.0)
  • Scenario: Mixed human + Claude Code editing session

Impact

This bug affects the accuracy of AI authorship statistics for any commits that:

  • Contain human-authored changes without explicit checkpoint tracking
  • Have invalid or missing attestation hashes in git notes
  • Mix human edits with AI-generated code in the same commit
Steps to reproduce

Steps to Reproduce

  1. Create a git repo with git-ai initialized
  2. Use Claude Code or similar to generate some lines (e.g., utils.ts with 20 lines)
  3. Manually edit another file and add 1 line (e.g., array.ts)
  4. Stage and commit both files together
  5. Run git-ai stats HEAD
  6. Observe that the commit is incorrectly marked as 100% AI instead of ~95% AI
Expected vs actual behavior

Expected Behavior

For a commit with:

  • 1 line human modification (manual edit)
  • 20 lines AI-generated (from Claude)

Expected: 5% human + 95% AI

Actual Behavior

Shows: 0% human + 100% AI

Root Cause

File: src/authorship/stats.rs
Function: accepted_lines_from_attestations()
Lines: 619-629

The function unconditionally counts all non-h_-prefixed attestation entries as AI modifications, regardless of whether the hash exists in metadata.prompts:

for entry in &file_attestation.entries {
    if entry.hash.starts_with("h_") {
        // Handle known-human modifications
        continue;
    }
    
    let accepted = /* calculate line count */;
    
    // ❌ BUG: Lines are counted as AI before checking if hash is valid
    total_ai_accepted += accepted;
    
    // This check only affects statistics grouping, not the count above
    if let Some(prompt_record) = log.metadata.prompts.get(&entry.hash) {
        let tool_model = format!("{}::{}", ...);
        *per_tool_model.entry(tool_model).or_insert(0) += accepted;
    }
}
Diagnostics (git-ai debug)
## Solution
Move the hash validation check BEFORE counting the lines as AI:


for entry in &file_attestation.entries {
    if entry.hash.starts_with("h_") {
        // Handle known-human modifications
        let accepted = /* calculate */;
        known_human_accepted += accepted;
        continue;
    }
    
    // ✅ FIX: Only count as AI if the hash exists in metadata
    if let Some(prompt_record) = log.metadata.prompts.get(&entry.hash) {
        let accepted = /* calculate */;
        total_ai_accepted += accepted;
        
        let tool_model = format!("{}::{}", ...);
        *per_tool_model.entry(tool_model).or_insert(0) += accepted;
    }
    // Otherwise, lines are not counted here and will be categorized as 
    // unknown_additions in stats_from_authorship_log()
}
Extra context (optional)

Dominant language
Rust
Stars
2.8k
Forks
296
Avg merge
1d 23h
Merged PRs (30d)
43

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from git-ai-project/git-ai

All issues in git-ai-project/git-ai

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.