Shared GitHub Actions workflow for integration tests across Harper repos
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
Research direction
Read the existing integration-tests GitHub Actions workflows in the oauth, generic plugin, nextjs, harper-pro, and harper repos first, comparing their shared and bespoke steps. Then assess the proposed .github/workflows/integration-tests.yml, .github/actions/setup/action.yml, and docs/ci.md; done means the two artifacts exist and oauth is converted as a proof of concept.
Written by the indexing model from the issue text.
Description
Idea
Most Harper repos that use the integration-testing framework have nearly identical integration-tests GitHub Actions workflows. Comparing five of them:
| Repo | Node default | Build | Fixtures | Shards | Bespoke extras |
|---|---|---|---|---|---|
| oauth | 22, 24 | ✅ | ✅ | — | none (pure simple tier) |
| generic plugin | 20/22/24 | optional | — | — | none |
| nextjs | 20, 22, 24 | ✅ | ✅ | — | playwright install, harper_ref override, trace artifacts |
| harper-pro | 22, 24, 26 | ✅ | — | 4 | submodules, re2 rebuild, legacy harperdb@4, concurrency cap |
| harper (core) | 22, 24, 26 | ✅ | — | 6 | Windows, Bun, uWS, legacy install |
The common skeleton — matrix generation → checkout → node setup → npm ci → build/fixtures → run test:integration → upload logs on failure — is identical across all of them. Everything that differs lives in the middle steps. That ~60% boilerplate is a good consolidation target; the bespoke middles are not.
Proposed approach: two shared artifacts, hosted here, versioned with the framework
The divergence lives mid-job, and a reusable workflow is all-or-nothing at the job level (you can't inject a step into its middle). So a single mechanism isn't enough — pair two:
-
A reusable workflow (
.github/workflows/integration-tests.yml,on: workflow_call) that owns the entire simple-tier pipeline (matrix gen, setup, run, logs-on-failure). Inputs:node-version/all-node-versions,build,install-fixtures,test-command,timeout-minutes. Callers shrink to ~8 lines. Covers oauth + any plain plugin. -
A composite action (
.github/actions/setup/action.yml) that owns just the shared setup steps (node +npm ci+ optional build/fixtures), dropped into a caller's own job. For repos that must interleave custom steps — nextjs (playwright,harper_ref), harper-pro/core (sharding, native rebuilds). The action deliberately does not checkout (harper-pro needssubmodules: recursive). Caveat: a composite action can't wrap a failure-triggered log upload around the caller's test step, so heavy-tier callers repeat those few lines.
Example: oauth's entire workflow becomes
name: Integration Tests
on:
push: { branches: [main] }
pull_request:
workflow_dispatch:
inputs:
node-version: { type: choice, default: all, options: [all, '22', '24'] }
jobs:
test:
uses: HarperFast/integration-testing/.github/workflows/integration-tests.yml@v1
with:
node-version: ${{ inputs.node-version || 'all' }}
all-node-versions: '[22, 24]'
build: npm run build
install-fixtures: npm run install:fixtures
timeout-minutes: 5
Example: nextjs / harper-pro keep their own job, reuse only setup
steps:
- uses: actions/[email protected]
# with: { submodules: recursive } # harper-pro
- uses: HarperFast/integration-testing/.github/actions/setup@v1
with:
node-version: ${{ matrix.node-version }}
build: npm run build
# --- project-specific middle: playwright / harper_ref / re2 / shards ---
- run: npx playwright install --with-deps chromium
- run: npm run test:integration
env: { HARPER_INTEGRATION_TEST_LOG_DIR: /tmp/harper-test-logs, FORCE_COLOR: '1' }
Where to draw the line
- Use the reusable workflow if your CI is just matrix → build → run → logs (oauth, most future plugins).
- Use only the composite action the moment you need a step in the middle (playwright,
harper_ref, sharding, native rebuilds) — keep your job, dedupe the setup. - harper core stays bespoke — its matrix (Windows × Bun × uWS × 6 shards × legacy install) is divergent enough that routing it through the shared workflow would bury it in conditionals. It can still open each job with the shared setup action.
Open questions / trade-offs to discuss
- Blast radius vs. drift. Consolidation trades N maintained files for one shared surface — a bad change can break every consumer's CI at once. Mitigate by pinning callers to a tag/SHA (never
@main) and versioning the workflow/action alongside the framework release, so CI config and framework move in lockstep. Is that coupling worth it vs. a copy-paste scaffold template (no runtime coupling, but drifts)? - Home. This repo (co-located + versioned with the framework) vs. a shared
HarperFast/.githubrepo. - Scope of inputs. How much variance (playwright toggle? shard count? legacy install?) belongs in the reusable workflow before it's better left to the composite action?
- Secrets. Callers add
secrets: inheritwhen tests need them — worth documenting as a convention.
If we pursue this, next step would be the two files + a short docs/ci.md adoption guide, then converting one simple consumer (oauth) as a proof of concept.
sent with Claude Opus 4.8
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from HarperFast/integration-testing
-
setupHarperWithFixture overwrites ctx.harper, dropping pre-set hostname (breaks multi-node add_node)Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
enhancement good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
Difficulty 3/5 1-2 days Newbie friendliness 74/100
-
A listener on all interfaces silently receives a test node's connections on macOS; the conflict canary cannot see itPossibly taken @dawsontoth claimed this 5 days ago. Open
HarperFast/integration-testing#38 · 1 comment · 1 assignee ·
-
Difficulty 4/5 3-5 days Newbie friendliness 58/100
All issues in HarperFast/integration-testing
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
keytrace logo svg?Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
HigherOrderCO/Bend#1294 ·
-
Price: 75 USD Priority: 2 (Medium) Time: <1 Hour
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day