Embedding varlock in npm-distributed CLIs (schema ships with the package)

Open
  • Is something wrong?
#1,258 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
28/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
node.js, typescript
Domain
cli, developer-experience

Research direction

Start by reading the existing multi-path loading entry point, loadVarlockEnvGraph, and trace how it handles schema files, telemetry, and errors. The issue proposes a new embedded API but leaves key design questions open, including resolver access and how user value files should layer. Done means an agreed API supports the stated CLI use case, with its security, error, and telemetry behavior defined.

Written by the indexing model from the issue text.

Description

core:varlock RFC

@lirantal you asked about this, so I'd like your feedback on the shape before we build anything.

Use case

A CLI distributed via npm ships its own .env.schema. End users of the CLI just write a .env (or set env vars), and those values get validated against the schema that came with the CLI. The schema is owned by the CLI author and trusted; the values are owned by the end user.

Goal: end users should not need to know varlock exists, create a schema, or install anything extra.

Proposal: first-class embedded API

import { createEmbeddedEnv } from 'varlock/embed';

const env = await createEmbeddedEnv({
  schemaPath: new URL('./.env.schema', import.meta.url),
  valueSources: ['./.env', '~/.config/mycli/.env'], // later wins, process.env on top
  valuesOnly: true,     // user files can set values but not define items/decorators/plugins
  unknownKeys: 'warn',
  errorFormatter: (err) => ..., // host-branded errors
});

env.API_KEY; // typed, validated, coerced

What this would handle for the CLI author:

  • Schema stays in the package. Plugins referenced by the schema resolve from the CLI's own node_modules, so global installs and npx work.
  • User files are values only. They can't add items, change types, or pull in plugins. Unknown keys warn (likely typos).
  • Host-branded errors. Validation failures read like the CLI's own errors, with no references to varlock commands.
  • No varlock telemetry from the CLI's end users.
  • Types generated at build time by the CLI author and shipped with the package.
  • Optional helper subcommands the CLI can expose almost for free, e.g. mycli env init (write a starter .env from the schema, with docs/examples as comments) and mycli env check.

Prototyping today

Multi-path loading already exists, so a rough version works now:

loadVarlockEnvGraph({ entryFilePaths: [`${pkgDir}/.env.schema`, `${process.cwd()}/.env`] })

It validates, but the user's .env is treated as a full env-spec file, errors mention varlock, and telemetry fires. Those gaps are what the embedded API would close.

(Alternatives like having the end user write their own schema that @import(npm:mycli/...)s the CLI's schema are possible but too clunky for this use case.)

Questions

  • Is your CLI project-scoped (reads .env from cwd), user-global (e.g. ~/.config/<cli>/), or both?
  • Should end users be able to use resolvers like op() / keychain() in their values? That's a big selling point for secrets, and means "values only" would mean "no schema changes" rather than "no function calls".
  • Do you want env-spec layering in the user's files (.env.local, .env.[env]), or just a flat values file?
  • Anything else in the integration that's been painful so far?
Dominant language
TypeScript
Stars
4.9k
Forks
140
Avg merge
2d 3h
Merged PRs (30d)
35

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from dmno-dev/varlock

All issues in dmno-dev/varlock

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.