docs(recipes): add a recipe for agent exploration of large codebases

Aperta
#1,185 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
57/100
Tipo di issue
Documentazione
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
github, markdown, typescript
Ambito
documentation

Direzione di ricerca

The destination is aidd_docs/recipes/; the issue says it currently contains only a .gitkeep. Read the existing recipe conventions and review the supplied recipe, especially its final Evidence link, version claims, and external links. Add the recipe with a short English slug, resolve the nonpublic benchmark reference, and verify the links and tool versions before merging.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary: Add an existing, field-tested recipe that helps agents answer precise questions on large or legacy codebases with verified evidence.

🌟 Objective / Problem

Finding reliable information and its source of truth in a large codebase is slow, and agents can answer confidently without proof. aidd_docs/recipes/ in the framework only holds a .gitkeep, so users have no reference for choosing and validating a retrieval approach.

🛠️ Proposed Solution

  1. Add the recipe below under aidd_docs/recipes/, with a short English slug.
  2. Fix its last line: the Evidence link points to a benchmark that is not public. Either publish a summary or remove the link.
  3. Recheck tool versions and external links before merging, since MCP tools change quickly.

🔍 Context / Technical Constraints

  • The recipe comes from a private repository, where it is being retired. It belongs in the framework rather than in operating notes.
  • The benchmark traces are not attached.

📎 Attachments / References

Full recipe: Make agent exploration efficient on large codebases
# Make agent exploration efficient on large codebases

Answer precise legacy-code questions with verified evidence.

## Why

- **Problem**: finding reliable information and its source of truth is slow.
- **Solution**: a dedicated research agent retrieves evidence you can verify.

## Steps to find and validate an answer

### 1) 🎯 Define the question

Choose a verifiable scenario.

1. **Describe**: inputs, behavior, exceptions.
2. **Freeze**: revision, local changes, searchable files/LOC, exclusions, languages, model/effort, tool versions.
3. **Prepare**: expected findings, required coverage, tolerated errors, maximum validation time. Hide expected findings from the agent.

```bash
git rev-parse HEAD
git status --porcelain=v1
opencode --version
```

#### Linux example

Example: [Linux v6.12](https://github.com/torvalds/linux/tree/v6.12), with a case-sensitive checkout.

```text
Audit an upload service against Linux v6.12. Research only; no patch.

Its dirfd is O_PATH|O_DIRECTORY for a trusted ext4 directory U.
It calls openat2(dirfd, path, {
  flags=O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC, mode=0600, resolve=P
}, sizeof(struct open_how)).

Compare separate policies:
B = RESOLVE_BENEATH|RESOLVE_NO_MAGICLINKS|RESOLVE_NO_XDEV
R = RESOLVE_IN_ROOT|RESOLVE_NO_MAGICLINKS|RESOLVE_NO_XDEV.

U contains safe/, rel -> safe, abs -> /safe, final -> safe/missing
(dangling), mnt/ (an outside directory bind-mounted from the same ext4
filesystem), and proc/ (procfs for the service's PID namespace).
N is the service's open fd for an outside directory.

Classify object or errno, and the first decisive kernel branch, for:
safe/new, rel/new, /safe/new, ../safe/new, abs/new, final, mnt/new,
proc/self/fd/N/new. Fresh fixtures: directories exist; new/missing do not;
permissions allow access; LSM/nosymfollow restrictions are absent.

Separate variants: remove NO_XDEV; remove O_EXCL for final;
add RESOLVE_CACHED; combine BENEATH and IN_ROOT. Explain errno ordering.

Then allow descendant renames, including moving a resolved directory
outside U. Trace RCU-to-REF retries versus userspace EAGAIN and
rename_lock/mount_lock checks around "..". Does confinement persist
during subsequent writes? Recommend bounded retries without weakening P.

Return an outcome matrix, call chain, v6.12 citations, and explicit limits.
```

### 2) ▶️ Run the OpenCode baseline

1. **Prepare**: install [OpenCode v2](https://opencode.ai/v2/docs/). Create `.opencode/agents/codebase-research.md` using this [agent format](https://opencode.ai/v2/docs/agents).

```markdown
---
description: Answers repository questions with checked evidence.
mode: primary
permissions:
  - { action: "*", resource: "*", effect: deny }
  - { action: read, resource: "*", effect: allow }
  - { action: glob, resource: "*", effect: allow }
  - { action: grep, resource: "*", effect: allow }
---
Read project memory and relevant module instructions.
Search only the declared corpus; no web or delegation.
Trace the scenario, dependencies, and exceptions.
Cite files:symbols:lines and revision.
Separate facts, hypotheses, conflicts, and missing evidence.
Report discovery methods; label additional methods as a mixed trial.
Do not edit code or propose patches.
```

2. **Reset**: before every baseline, restore this permission list; set all MCP servers' `disabled` to `true`. Reload configuration.
3. **Launch**: authenticate; replace the example with an available model.

```bash
opencode auth login
opencode models
RESEARCH_MODEL='openai/gpt-6.1-sol#max'
opencode reload
opencode mini --agent codebase-research --model "$RESEARCH_MODEL"
```

4. **Check**: require `read` and `grep` calls on a known file (`MAINTAINERS` for Linux).
5. **Measure**: fresh session, no `--continue`; submit question/corpus/revision and start timing.

### 3) 🧭 Choose one alternative

Match the observed gap; prefer an existing setup.

| Priority | Approach / test | Use when… | Hosting | Open source? | Requirement |
| --- | --- | --- | --- | --- | --- |
| **Start** | [OpenCode](https://opencode.ai/v2/docs/tools) | Establishing a baseline | Local search; local/cloud model | [Yes](https://github.com/anomalyco/opencode/blob/dev/LICENSE) | Supported model |
| **Symbols** | [Serena](https://oraios.github.io/serena/) · [Test](#serena) | Definitions/references are missing | Local MCP | [Yes](https://github.com/oraios/serena/blob/main/LICENSE) | LSP or JetBrains setup |
| **Symbols — Claude option** | [Claude LSP](https://code.claude.com/docs/en/plugins/code-intelligence) · [Test](#lsp) | You use Claude Code | Local LSP; cloud model | [clangd yes](https://llvm.org/docs/DeveloperPolicy.html#new-llvm-project-license-framework); Claude proprietary | Language server |
| **Concepts** | [Augment](https://docs.augmentcode.com/context-services/mcp/overview) · [Test](#augment) | Names differ from question vocabulary | Local/remote MCP; [cloud code storage](https://docs.augmentcode.com/cli/setup-auggie/workspace-indexing#security-and-privacy) | [No](https://github.com/augmentcode/auggie/blob/main/LICENSE.md) | Account, usage billing |
| **Remote repos** | [Sourcegraph](https://sourcegraph.com/docs/api/mcp) · [Test](#sourcegraph) | Indexed remote revisions matter | [Self-hosted/cloud](https://sourcegraph.com/docs/self-hosted/deploy) | [Enterprise proprietary](https://sourcegraph.com/terms/tos#3-proprietary-rights-and-licenses) | Enterprise access |
| **Another agent** | [Cursor ACP](https://cursor.com/docs/cli/acp) · [Test](#cursor-through-acp) | Comparing another agent in your IDE | Local client; [cloud AI](https://prod.cursor.com/help/security-and-privacy/privacy) | [Proprietary](https://cursor.com/en-US/terms-of-service); ACP open | Cursor account |
| **Last** | [Qdrant](https://github.com/qdrant/mcp-server-qdrant/blob/v0.8.1/README.md) · [Test](#qdrant-rag-pilot) | Controlling the indexed corpus | Local here; [self-hosted/cloud available](https://qdrant.tech/documentation/overview/) | Yes: [database](https://github.com/qdrant/qdrant/blob/master/LICENSE), [MCP](https://github.com/qdrant/mcp-server-qdrant/blob/v0.8.1/LICENSE) | Ingestion, freshness |

**Scope**: OpenCode v2.0.28; [native LSP is unsupported in v2](https://opencode.ai/v2/docs/migrate-v1/#supported-fields-without-direct-native-equivalents). Hosting concerns retrieval; [model context](https://opencode.ai/v2/docs/models) may still leave your machine. Priorities guide selection, not performance.

### 4) 🔎 Test the alternative

Reuse question, corpus, revision, instructions, and time budget. Keep the same model/effort within each OpenCode or Claude comparison.

1. **Configure**: follow the chosen procedure.
2. **Check**: verify a source result; log actual calls. An unused tool provides no evidence of benefit.
3. **Measure**: fresh session; submit question/tool instruction and start timing.

**Isolation**: inspect calls. Outside-corpus/web evidence invalidates the run; extra discovery methods make it mixed. Claude/Cursor comparisons measure complete agents; record their models.

#### MCP trial startup

After registering the chosen server:

1. **Expose**: set `"codemode": false` in its `opencode.json` entry. [Direct tools](https://opencode.ai/v2/docs/mcp-servers#permissions)
2. **Restrict**: restore baseline permissions; allow only chosen actions below. Enable that server; disable others.
3. **Reload**: one shared server; one Qdrant MCP process per database. Require a tool call.

| Tool | Actions to allow |
| --- | --- |
| Serena | `serena_initial_instructions`, `serena_find_symbol`, `serena_find_referencing_symbols`, `serena_get_symbols_overview`, `serena_search_for_pattern` |
| Augment | `augment_codebase-retrieval` |
| Sourcegraph | `sourcegraph_code_finder`, `sourcegraph_read_file` |
| Qdrant | `qdrant_qdrant-find` |

Append entries in this form:

```yaml
- { action: augment_codebase-retrieval, resource: "*", effect: allow }
```

```bash
opencode reload
opencode mcp list
opencode mini --agent codebase-research --model "$RESEARCH_MODEL"
```

#### Serena

1. **Install**: [uv](https://docs.astral.sh/uv/getting-started/installation/) and [Serena](https://oraios.github.io/serena/02-usage/010_installation.html).
2. **Connect**: use its [IDE context](https://oraios.github.io/serena/02-usage/030_clients.html#terminal-based-clients).

```bash
uv tool install -p 3.13 serena-agent
serena init
opencode mcp add serena -- serena start-mcp-server \
  --context ide --project-from-cwd --mode no-onboarding --agent-interface tools
```

3. **Check**: call `find_symbol` and `find_referencing_symbols` on known cross-file symbols after indexing. Empty results do not prove absence.
4. **Ask**: “Use Serena symbols and references; report missing coverage.”

C/C++ requires [project setup](https://oraios.github.io/serena/03-special-guides/cpp_setup.html). [JetBrains alternative](https://oraios.github.io/serena/02-usage/025_jetbrains_plugin.html): install its plugin, open the same root, add `--language-backend JetBrains`, reconnect, check the dashboard.

#### LSP

1. **Prepare**: install [Claude Code](https://code.claude.com/docs/en/overview) and [clangd](https://clangd.llvm.org/installation); configure [compile commands/headers](https://clangd.llvm.org/design/compile-commands). For Linux, generate them in a Linux environment; record architecture/configuration and covered translation units.
2. **Create**: `.claude/agents/codebase-research.md` with this [header](https://code.claude.com/docs/en/sub-agents), followed by the shared instruction body.

```yaml
---
name: codebase-research
description: Answers repository questions with checked evidence.
model: inherit
permissionMode: plan
disallowedTools: Write, Edit, NotebookEdit, Agent, WebSearch, WebFetch
---
```

3. **Baseline**: freeze an available model ID; run native Claude first.

```bash
CLAUDE_RESEARCH_MODEL=claude-sonnet-5-5
```

```bash
CLAUDE_CODE_DISABLE_AUTO_MEMORY=1 claude \
  --model "$CLAUDE_RESEARCH_MODEL" --agent codebase-research --permission-mode plan \
  --strict-mcp-config --mcp-config '{"mcpServers":{}}' \
  --disallowedTools "LSP"
```

4. **Enable**: [install the plugin](https://code.claude.com/docs/en/plugins/install) in local scope; reload.

```bash
claude plugin install clangd-lsp@claude-plugins-official --scope local
```

```bash
CLAUDE_CODE_DISABLE_AUTO_MEMORY=1 claude \
  --model "$CLAUDE_RESEARCH_MODEL" --agent codebase-research --permission-mode plan \
  --strict-mcp-config --mcp-config '{"mcpServers":{}}'
```

5. **Ask**: “Use LSP definitions/references; report missing coverage.”

Keep the [indexed server](https://clangd.llvm.org/design/indexing) alive. Check coordinates and known cross-file references; successful calls can be incomplete. Failed setup: “not configured.”

#### Augment

1. **Prepare**: [Node.js 20+, Augment account/billing](https://docs.augmentcode.com/cli/setup-auggie/install-auggie-cli).
2. **Connect**: use the [OpenCode integration](https://docs.augmentcode.com/context-services/mcp/quickstart-open-code).

```bash
npm install -g @augmentcode/auggie@latest
auggie login
opencode mcp add augment -- auggie --mcp --mcp-auto-workspace
```

3. **Ask**: “Use codebase-retrieval; set directory_path to my absolute checkout path.”

Preflight retrieval: our free account returned `402 Payment Required`. Record [indexing time/exclusions](https://docs.augmentcode.com/cli/setup-auggie/workspace-indexing).

#### Sourcegraph

1. **Prepare**: [Enterprise MCP access](https://sourcegraph.com/docs/api/mcp), indexed repositories/SHA, `code_finder` quota.
2. **Connect**: replace the example domain; [authenticate](https://sourcegraph.com/docs/api/mcp/client-integrations#opencode).

```bash
opencode mcp add sourcegraph --url https://sourcegraph.example.com/.api/mcp
opencode mcp auth sourcegraph
```

3. **Ask**: “Call code_finder with the repository in task and SHA in revision. Verify with read_file at the same SHA.”

Repeat per known repository; report missing access/revisions. Unpushed changes are unavailable.

#### Cursor through ACP

1. **Prepare**: [JetBrains 2025.3.2+ and AI Assistant](https://blog.jetbrains.com/ai/2026/03/cursor-joined-the-acp-registry-and-is-now-live-in-your-jetbrains-ide/), [paid Cursor](https://cursor.com/docs/integrations/jetbrains).
2. **Connect**: AI Chat → Install From ACP Registry → Cursor. Authenticate.
3. **Isolate**: disable [Pass custom MCP servers / Pass IntelliJ MCP server](https://www.jetbrains.com/help/ai-assistant/acp.html) and Cursor's optional MCP servers.
4. **Ask**: confirm [ACP Ask mode](https://cursor.com/docs/cli/acp); paste the shared instruction body. Add “Use native code search.”

If Ask is unavailable, [install Cursor CLI](https://cursor.com/docs/cli/installation) and use [CLI Ask](https://cursor.com/docs/cli/overview):

```bash
cursor-agent --version
cursor-agent login
cursor-agent --mode=ask
```

Record interface, model, calls, backend. Use [ACP logs](https://www.jetbrains.com/help/ai-assistant/acp.html#collect-acp-logs) if needed; full logs require `llm.agent.extended.logging` and IDE restart. Unidentified [Instant Grep](https://cursor.com/docs/agent/tools/search) means “backend unknown.”

#### Qdrant RAG pilot

1. **Freeze**: representative files/revision; rerun native search on exactly that corpus.
2. **Ingest**: install [uv](https://docs.astral.sh/uv/getting-started/installation/). Save the [tested C/H script](../benchmarks/linux-retrieval/ingest_pilot.py) as `.tmp/ingest_pilot.py`. This Linux example selects every top-level `fs/*.c` and `fs/*.h`.

```bash
uv run --no-project --python 3.11 --with mcp-server-qdrant==0.8.1 \
  python .tmp/ingest_pilot.py --corpus fs --revision "$(git rev-parse HEAD)" \
  --database .tmp/legacy-rag-pilot/db --collection legacy-pilot-v1 \
  --cache-dir .tmp/legacy-rag-pilot/cache \
  --output .tmp/legacy-rag-pilot/ingestion.json
```

Verbatim chunks: ≤200 content tokens, 20-token overlap; path prefix included within the model's 256-token limit. Check stored files/chunks and source coverage. Use a fresh database.

3. **Connect**: after ingestion closes, register the [pinned MCP](https://github.com/qdrant/mcp-server-qdrant/blob/v0.8.1/README.md) read-only.

```bash
opencode mcp add qdrant \
  --env QDRANT_LOCAL_PATH="$(pwd)/.tmp/legacy-rag-pilot/db" \
  --env COLLECTION_NAME=legacy-pilot-v1 \
  --env EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2 \
  --env FASTEMBED_CACHE_PATH="$(pwd)/.tmp/legacy-rag-pilot/cache" \
  --env QDRANT_SEARCH_LIMIT=5 --env QDRANT_READ_ONLY=true \
  -- uvx --python 3.11 [email protected]
```

4. **Ask**: “Use qdrant-find; verify original files/revisions. Report missing evidence; check the file manifest before declaring a source absent. No corpus expansion or fallback.”

Storage/embeddings are local; the model downloads initially. [QDRANT_URL/QDRANT_API_KEY must be absent](https://github.com/qdrant/mcp-server-qdrant/blob/v0.8.1/src/mcp_server_qdrant/settings.py). Only `information` is embedded; no scanning, synchronization, or deduplication. Record ingestion separately.

### 5) ⏱️ Compare validated results

Choose the simplest approach passing your criteria.

1. **Validate**: score the first answer's findings, exceptions, source support and false/unsupported extras. Time review and corrections separately. Use the same criteria and constraints.
2. **Measure**: capture [`opencode run --format json`](https://opencode.ai/v2/docs/cli/commands/#run) calls; [`opencode session export SESSION_ID`](https://opencode.ai/v2/docs/cli/commands/#session) gives session traces/usage. Claude uses [`/usage`](https://code.claude.com/docs/en/costs#using-the-usage-command).
3. **Repeat**: more questions; separate setup/indexing, first/repeated queries, and review. Count warmup in first-answer time. Enforce wall-clock deadlines; flag sleep and API retries. Record retrieval charges; unknown cost is “N/A.”

| Approach / model / methods | Validated / expected findings | False / unsupported claims | Research / review | Setup / indexing | Cost / usage |
| --- | --- | --- | --- | --- | --- |
| Baseline | Fill in | Fill in | Fill in | Fill in | Fill in |
| Alternative | Fill in | Fill in | Fill in | Fill in | Fill in |

**Source conflicts**: implemented behavior → code/tests at revision; deployed behavior → runtime configuration too. Report conflicting documentation.

#### Linux validation key

Evaluator only: keep this key outside the agent's prompt and corpus.

| Path, without races | B | R |
| --- | --- | --- |
| safe/new | Creates U/safe/new | Creates U/safe/new |
| rel/new | Creates U/safe/new | Creates U/safe/new |
| /safe/new | EXDEV | Creates U/safe/new |
| ../safe/new | EXDEV | Creates U/safe/new |
| abs/new | EXDEV | Creates U/safe/new |
| final | EEXIST | EEXIST |
| mnt/new | EXDEV | EXDEV |
| proc/self/fd/N/new | EXDEV at mount | EXDEV at mount |

Check [path resolution](https://github.com/torvalds/linux/blob/v6.12/fs/namei.c#L1966), [exclusive open](https://github.com/torvalds/linux/blob/v6.12/fs/namei.c#L3730), and [selftests](https://github.com/torvalds/linux/blob/v6.12/tools/testing/selftests/openat2/resolve_test.c#L136).

- **Flags**: BENEATH + IN_ROOT → EINVAL; O_CREAT + RESOLVE_CACHED → EAGAIN before lookup. [build_open_flags](https://github.com/torvalds/linux/blob/v6.12/fs/open.c#L1215)
- **Variants**: no NO_XDEV → outside bind target; proc magic-link → ELOOP. No O_EXCL on final → creates U/safe/missing. [mounts](https://github.com/torvalds/linux/blob/v6.12/fs/namei.c#L1552), [symlink handling](https://docs.kernel.org/6.12/filesystems/path-lookup.html#following-the-symlink-in-the-final-component)
- **Creation**: `do_open` skips `complete_walk` after `FMODE_CREATED`. A resolved directory moved outside U can receive the new file. [Creation branch](https://github.com/torvalds/linux/blob/v6.12/fs/namei.c#L3739)
- **Writes**: the fd retains the file; `vfs_write` does not reapply P. [Write path](https://github.com/torvalds/linux/blob/v6.12/fs/read_write.c#L663)
- **Races**: internal ECHILD/ESTALE retries differ from userspace EAGAIN. No fixed errno for every race; resolution does not freeze topology. Bound retries without dropping confinement flags. [do_filp_open](https://github.com/torvalds/linux/blob/v6.12/fs/namei.c#L3952), [complete_walk](https://github.com/torvalds/linux/blob/v6.12/fs/namei.c#L883), [locking](https://docs.kernel.org/6.12/filesystems/path-lookup.html#rename-lock)

## Verify

- [ ] **Answer validated**: findings, exceptions, uncertainties.
- [ ] **Evidence checked**: selected tool used; precise sources, correct revision.
- [ ] **Effort recorded**: review time, setup, cost if available.

**Evidence**: [Linux benchmark: results, limits and traces](../benchmarks/linux-retrieval/README.md). Official docs, CLI calls and v6.12 sources checked October 14, 2026. Research in sources does not prove runtime behavior.

🙏 QA

  • The recipe renders correctly and every link resolves.
  • Its Verify checklist can be followed end to end on a sample repository.
Lingua principale
TypeScript
Stelle
481
Fork
45
Merge medio
1g 13h
PR unite (30g)
91

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di ai-driven-dev/framework

Tutte le issue di ai-driven-dev/framework

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.