Add a command search so agents can find commands and flags by keyword

Open
  • Is something wrong?
#1,161 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
45/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
rust
Domain
cli, developer-experience

Research direction

Start by reading the clap command-tree construction and the root help’s CONTEXT FOR AGENTS: block. Add the search entry point, then write clap parsing tests and ranking unit tests against the real command tree. Done means the command returns ranked leaf matches with --json, and the agent evaluation reduces calls or tokens without lowering success.

Written by the indexing model from the issue text.

Description

enhancement

Problem

Agents learn the CLI by walking --help one level at a time. That is cheap when the agent guesses the right branch, and fails when the capability is a flag rather than a command.

Measured on main (f3698472): 279 help pages, 223 of them leaf commands, about 350 KB of help in total (~87k tokens). An average leaf is 3.45 levels deep and an average page is ~1.25 KB, so a correct walk costs about 1k tokens over 3–4 calls. A full machine-readable catalog would be too large to dump, and even a leaf index of paths and about lines is ~3.3k tokens.

The walk breaks down on flags. Examples:

  • "IP access list" is --ip-allow on cloud service create, --add-ip-allow/--remove-ip-allow on cloud service update, and "access rules" on cloud postgres update. No command name or about line leads there.
  • "idle" appears in the help of 7 leaf commands but in only 2 about lines.

Proposal

A command that searches the help tree by keyword and returns the few best matches. Name to decide: clickhousectl help search would clash with clap's built-in help <command>, and a top-level find reads as "find a resource", so its about must say it finds commands. Prior art: az find, gcloud help -- TERMS.

  • Build the index at runtime from the clap command tree, so it can never go stale and needs no generated file.
  • Match command paths, about lines, flag names, flag help and CONTEXT FOR AGENTS: blocks. Add a small synonym table for console vocabulary that the CLI names differently (for example "IP access list" → ip-allow).
  • Return the top ~5 leaf commands, each with its path, about and the matching flag lines. Target ~150 tokens for a typical result. Support --json.
  • Point to it from the root help's CONTEXT FOR AGENTS: block.

Evaluation

Measure before and after with an agent eval: a fixed set of tasks whose answer is a specific command and flags, including flag-only capabilities like the examples above. Record success, tool calls and help tokens read until the agent runs the right command. The same harness can serve #915.

Done when

  • The search command exists, with clap parsing tests and ranking unit tests over the real command tree (structural assertions, not wording pins).
  • The eval shows fewer calls or tokens to reach the right command, without lower success.
Dominant language
Rust
Stars
80
Forks
9
Avg merge
3d 20h
Merged PRs (30d)
57

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ClickHouse/clickhousectl

All issues in ClickHouse/clickhousectl

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.