Epic: Move service selection into the backend API instead of pre-filtering the project in the CLI
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 15/100
Research direction
This is a design epic, not a task to pick up. Start by reading how WithSelectedServices is applied in cmd/compose/up.go and cmd/compose/scale.go, then the reconciler's servicesUsingNetwork and servicesUsingVolume helpers. Done means maintainers agree on the ServiceSelection shape and the open questions in the issue are resolved; implementation work would follow in separate issues.
Written by the indexing model from the issue text.
Description
"This issue was filed by an AI agent on a human's behalf. The human submitter may not have independently verified the report."
Summary
Service selection (up web, --no-deps, watch rebuilds, …) is currently applied by the callers of the backend, by pre-filtering the types.Project with WithSelectedServices. This issue proposes making the selection an explicit input of the backend API, so that the reconciler sees the whole model and decides what the selection means. It also matters for other consumers of pkg/api (library users, Docker Desktop, …): they should not have to reproduce the CLI's pre-filtering to get correct behavior.
Related: #14081 (converging the start phase into the plan engine), #14291.
Problem
Two different notions of "selection" coexist today:
Servicesin the API options (CreateOptions.Services, …) means targeted services. The reconciler still converges every other service, with theRecreateDependenciespolicy (reconcileService).- Pre-filtering the project (
project.WithSelectedServices(...), done incmd/compose/up.goandcmd/compose/scale.gofor--no-deps, and inwatch's rebuild) means the other services do not exist: they are moved toDisabledServices, andDependsOnof the kept services is pruned to the selection.
Because the reconciler receives a truncated model without knowing it, it makes wrong inferences. Concretely, it looks for the consumers of a network or volume in project.Services only (servicesUsingNetwork, servicesUsingVolume). With a narrowed project, a drifted network or volume that is also used by a service outside the selection looks unused by it:
- the plan stops/disconnects (or removes) only the selected services' containers, then tries to remove the network/volume, which fails with "in use";
- for a volume, the user is first asked to confirm a data-destroying recreation that cannot succeed.
I reproduced this at the reconciler level with unit tests, on a project narrowed to one service whose network (or volume) is shared with a disabled service that has a live container. up --no-deps <svc> and scale --no-deps build exactly such a project, so I expect the same exposure there; I have not reproduced it end to end against a daemon. Dependency pruning in WithSelectedServices may also affect config hashes; I have not checked that.
A narrow guard in the reconciler (leave a drifted resource alone, with a warning, while a disabled service's live container still holds it) is being added to #14291 for the watch rebuild. It is a mitigation: it works around the truncated model rather than removing the cause.
Proposal
Keep the full project as the model and pass the selection to the backend as data. A sketch, open for discussion:
// ServiceSelection scopes an operation to part of the project.
type ServiceSelection struct {
Services []string
Dependencies DependencyScope // include (default), ignore (--no-deps), dependents (watch rebuild)
}
carried by the options of the operations that converge or start containers (CreateOptions, UpOptions/StartOptions, ScaleOptions, WatchOptions), and applied at a single point, in the plan preparation, once the canonical project has been built.
Semantics the reconciler would own:
- The model stays whole: every consumer of a network or volume is visible, and the dependency graph is the real one.
- The selection bounds which services' containers may be created, recreated, started, stopped or waited on.
- Infrastructure needed by the selected services is created if missing. A drifted network or volume is recreated only if all its consumers are within the selection; otherwise it is kept, with a warning.
- Orphan detection is unaffected by the selection.
Compatibility
- Additive: a caller that keeps pre-filtering the project still works, since it just yields a project with fewer services.
- The existing
Servicesfields keep their current meaning ("targets"); the new field must not silently redefine them. - Commands that never go through the plan (
config,pull,push,run's own container) can keep pre-filtering client side. Migration can go command by command, starting withup,create,start,scaleandwatch.
Questions
- Is a single
ServiceSelectionvalue the right shape, or should the dependency scope stay a separate option? - Should
RecreateDependenciesbe folded into the selection, since it exists to express "what happens to services outside the targets"? - Are there library consumers relying on the current pre-filtering contract that this would break?
Feedback from anyone consuming pkg/api is welcome.
- Dominant language
- Go
- Stars
- 38.5k
- Forks
- 5.9k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 66
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from docker/compose
-
[BUG] deploy.restart_policy without condition drops the restart policyPossibly taken @akshat-lakhera claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
docker/compose#14383 · 1 comment ·
Maintainers usually reply within 1 day
-
[BUG] `depends_on` health/completion waits always sleep 500ms before the first checkPossibly taken @htoyoda18 claimed this 1 day ago. Openkind/bug status/0-triage
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
[BUG] down -v removes a named volume owned by another projectPossibly taken @htoyoda18 claimed this 4 days ago. Openkind/bug
Difficulty 3/5 1-2 days Newbie friendliness 73/100
Maintainers usually reply within 1 day
-
[BUG] A start failure in one service aborts the start of unrelated services (unreleased regression)Openkind/bug
Difficulty 4/5 3-5 days Newbie friendliness 57/100
docker/compose#14372 · 1 comment ·
Maintainers usually reply within 1 day
-
kind/feature
Difficulty 4/5 3-5 days Newbie friendliness 48/100
docker/compose#14365 · 2 comments ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
blinklabs-io/dingo#5636 ·
Maintainers usually reply within 1 day
-
automated-analysis bug cookie safe-outputs tool-improvement workflow-issue
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
github/gh-aw#71952 · 3 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
JetBrains/youtrack-cli#130 ·
Maintainers usually reply within 1 day
-
Cloudflare Pages secret sync throws TypeError when deployment_configs[environment] is undefinedOpen
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Infisical/infisical#8892 · 1 comment ·
Maintainers usually reply within 1 day
-
windvane requests one more page than it needsPossibly taken @TassioSales claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
projectdiscovery/subfinder#1979 ·
Maintainers usually reply within 3 days