Name live data source code consistently: Live, LiveIntegration and LiveSource

Open
#32,679 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Refactor
Clarity
Clearly specified
Activity status
Active
Tech stack
typescript

Research direction

Read the naming table and Phase 1 list, then trace the listed TypeScript types, variables, and live integration file; the issue does not name their paths. Start with the non-breaking renames and check that table names and metric names stay unchanged. Done means Phase 1 is merged and new code follows the naming table, with a plan for each locked Phase 2 name.

Written by the indexing model from the issue text.

Description

Why

Live data source code uses several names for the same concepts. One configured source is called LiveSource*, ApiSource*, ExternalConnectionApiSource and apiSource depending on the file. Integration-level things are sometimes named after a source (LiveSourceSetupKind, LIVE_SOURCE_CATALOG). Readers can't tell from a name what it belongs to.

Target naming

Prefix Means Example
Live* the shared engine, used by every integration LiveCalls, LiveHttpRequest
LiveIntegration* one integration kind (GA4, PostHog) LiveIntegrationType, LIVE_INTEGRATION_CATALOG
LiveSource* one source configured in a project LiveSourceFields, LiveSourceSettings
ExternalConnection* only the generic storage and auth container ExternalConnectionModel
ApiSource*, ApiQuerySource* legacy; kept only where renaming breaks something ApiSourceExplore (swagger)

No new ApiSource* names, and no ExternalConnection* names in live engine code.

Phase 1: non-breaking (internal TypeScript only)

  1. Integration-level names: LiveSourceSetupKind, LiveSourceCatalogEntry, LiveSourceAgentQueryGuide, the frontend LIVE_SOURCE_CATALOG, and the file liveSourceIntegrations.ts move to LiveIntegration*.
  2. Backend ApiSource* names that aren't in swagger: the call limiter, call and lease types, table type constants (table names unchanged), metric variables (metric names unchanged), ApiSourceQueryContext/Issue, and ApiSourceOf<C> (to LiveSourceSettings<C>).
  3. Live engine code that says ExternalConnection* when it means a source's settings uses LiveSourceSettings.
  4. Then the apiSource variables (about 570 uses), once the types above have landed.

Phase 2: breaking (later, planned with release safety)

  • Swagger schemas: ApiSourceExplore, ApiSourceQuery, ApiSourceCapabilities, ApiSourcePushdown, ApiSourcePayload, ApiSourceSchemaScan, ApiSourceSchemaEvent, ApiSourceFieldRule, ApiSourceIncompatibility, ApiSourceMetricAggregation, ExternalConnectionApiSource, ExternalConnectionApiUsage.
  • The stored connector key on each source's settings (needs a migration and an API that accepts both).
  • Metric, trace, job and analytics event names (apiConnector, lightdash.apiConnector, api_source_* metrics), plus the product analytics charts that read them.
  • Table names and ApiQuerySource*.

Done when

  • Phase 1 is merged, and new code follows the table above (it's written into the add-live-integration skill).
  • Phase 2 has a plan for each locked name: rename with compatibility, or keep with a reason.

Follows the live integration redesign in #32675. An earlier, broader attempt (SPK-2906) was cancelled.

Dominant language
TypeScript
Stars
6.2k
Forks
809
Avg merge
12h 13m
Merged PRs (30d)
854

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from lightdash/lightdash

All issues in lightdash/lightdash

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.