ckanext-ecosystem: a phone-home extension for the catalog

Context

The catalog currently learns about the world by looking inward-out: weekly crawls hit each registered site's status_show, package_search, group_list and organization_list endpoints and record what comes back. That works, but it is lossy and fragile — status_show returns a bare list of extension names with no versions, sites behind Cloudflare or with broken TLS need scraper workarounds, and nothing tells us when a site's operator wants their entry updated.

A small CKAN extension installed on participating sites inverts this. The site tells the catalog what it is running, on its own schedule, accurately.

This issue is the design and scope agreement, not the implementation. ckanext-ecosystem has no repository yet; sub-issues get filed once one exists and the scope below is agreed.

Proposed scope

• Self-registration / metadata sync. An opted-in site updates its own catalog entry from the running instance, removing the crawl dependency for those sites.
• Precise version capture. CKAN version plus per-extension versions — richer than the bare extension-name list status_show gives us today.
• Upgrade alerts. Notify a site operator when an extension they run has a newer release. This is the inside-out counterpart to #177, which proposes watching extension repos from the catalog side; the two together cover both directions.
• Federated search. Search across participating sites that expose a DCAT endpoint. Depends on knowing who exposes one — see #197.
• Auth model. How does a site prove it owns the catalog entry it is updating? Related to #130 (access control) — a site pushing metadata is a write path into the catalog and needs the same answer.

Out of scope for this issue

Writing the extension. Agree the scope here, then create the repo.

Open questions

• Does the extension push to the catalog, or expose an endpoint the catalog pulls from? Push is simpler for sites behind firewalls; pull keeps the catalog in control of scheduling and is harder to abuse.
• Do we keep crawling sites that install the extension, as a cross-check, or trust the self-report?
• What is the minimum CKAN version we target?