Use persisted Maven metrics for repository /metrics/ responses

Open
  • Is something wrong?
#568 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
45/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
api, databases

Research direction

Start by reading the current Maven /metrics/ action and the combined aggregate described in pulp_maven#541; the issue does not name source files or tests. Decide how completed repository versions get their persisted metrics and how existing versions and null counts are handled. Done means /metrics/ reads persisted counts with current semantics, while explicit repository-version requests, removed content, and version isolation remain correct.

Written by the indexing model from the issue text.

Description

Proposed issue

Title

Use persisted Maven metrics for repository /metrics/ responses

Problem

Content Sources periodically calls:

GET /api/pulp/lightwell/api/v3/repositories/maven/maven/<uuid>/metrics/

For lightwell/java/validated, this request took several minutes and timed out. CloudWatch showed three Content Sources API pods issuing the same request concurrently.

The endpoint currently calculates three distinct aggregates at request time:

package_count: distinct (group_id, artifact_id)
version_count: distinct (group_id, artifact_id, base_version)
build_count:   distinct (group_id, artifact_id, version)

This remains expensive for large Maven repositories.

Why existing optimizations are insufficient
  • pulp_maven#541 combines the three aggregates into one SQL query. This removes query round trips but still scans and aggregates the repository's MavenPackage membership on every cache miss.
  • pulpcore#8169 uses persisted repository-version summaries for unfiltered content-list pagination counts. It does not affect the Maven custom /metrics/ action.
  • Generic pulpcore summaries do not contain all required Maven values. They can represent MavenPackage content-unit count, which is equivalent to build_count, but not distinct GA or normalized base-version counts.
Proposed change

Add a MavenRepositoryVersionMetrics model linked one-to-one to RepositoryVersion:

repository_version
package_count
version_count
build_count
calculated_at

Calculate and persist these values once for each completed repository version, using the combined aggregate introduced by #541. Update the Maven /metrics/ endpoint to read the persisted row instead of recomputing counts.

Question: should we retain fallback count query if persisted counts null for some reason?
Question: should we schedule a task to backfill existing completed versions or do this in a migration?

Acceptance criteria
  • /metrics/ performs an indexed lookup of persisted counts for completed versions.
  • Count values preserve current API semantics exactly.
  • Explicit repository_version requests continue to work.
  • Removed content and repository-version isolation remain correct.
Dominant language
Python
Stars
5
Forks
30
Avg merge
1d 16h
Merged PRs (30d)
26

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pulp/pulp_maven

All issues in pulp/pulp_maven

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.