bug: OTEL_EXPORTER_OTLP_ENDPOINT in the environment breaks trigger deploy (depot build)

未关闭
#4,321 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

  • #4370 来自 @eeshsaxena —— 已关闭,未合并

评估

难度
3/5
预计耗时
1-2 天
新手友好度
45/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
停滞
领域
devops

调研方向

从 packages/cli-v3/src/deploy/buildImage.ts 中的 depot 子进程调用入手,其中传给 depot() 的 env 对象通过 execa 的 extendEnv 合并到 process.env 上。在提出如何为 depot 子进程剥离或中和 OTEL_EXPORTER_OTLP_* 同时保持 PATH/HOME 不变的方案之前,先阅读已关闭且未合并的 PR #4370,看看维护者拒绝了哪种做法。完成的标准是:设置了 OTEL_EXPORTER_OTLP_ENDPOINT 的 deploy 能完成远程 depot 构建,且 --local-build 路径仍然可用。

由索引模型根据 Issue 内容生成。

描述

Provide environment information
System:
  OS: macOS 15 (also reproduced on Linux, GitHub Actions ubuntu-24.04)
Binaries:
  Node: 22.x
  npm / pnpm / bun: any
trigger.dev CLI 4.5.4, @trigger.dev/sdk 4.5.4, @trigger.dev/build 4.5.4
@depot/cli 0.0.1-cli.2.80.0 (pinned by trigger.dev 4.5.4)
Describe the bug

In plain terms: OTEL_EXPORTER_OTLP_ENDPOINT is a normal, standard
environment variable that lots of projects already set — it tells OpenTelemetry
where to send traces. If that variable is set when you run trigger deploy, the
deploy fails while building the image. Remove the variable and the exact same
project deploys fine.

That's the bug: a value the user put in their own environment, for their own
unrelated tooling, breaks trigger.dev's build. A user-set environment variable
shouldn't be able to do that. The deploy should either ignore it or keep it out
of the build, not fall over on it. The error messages also point at "depot" and
look like an outage, so it's easy to lose a lot of time blaming the wrong thing
(we did) instead of realizing one of your own env vars is the cause.

Here's what actually happens under the hood.

If OTEL_EXPORTER_OTLP_ENDPOINT is set in the environment, trigger deploy
fails during the remote (depot) image build, before the build really starts.
The variable's value only changes which error you get:

  • a valid URL (e.g. http://localhost:4318) →
    Error: cannot merge resource due to conflicting Schema URL
  • any non-URL value (in our case a dotenvx encrypted:… blob) →
    failed to build resolver: passthrough: received empty target in Build()

Both surface as Error building image, with that line as the last log. Unset
the variable and the same project deploys fine on the same depot backend
seconds later. --local-build is unaffected.

We chased this for a while as flaky depot infrastructure (the empty target
error reads like a depot outage). It isn't — it's deterministic per-environment,
and it's the OTLP endpoint variable leaking into the depot build.

Root cause. The remote build runs the depot CLI via
packages/cli-v3/src/deploy/buildImage.ts:

const childProcess = depot(args, {
  cwd: options.cwd,
  env: {
    DEPOT_BUILD_ID: options.buildId,
    DEPOT_TOKEN: options.buildToken,
    DEPOT_PROJECT_ID: options.buildProjectId,
    DEPOT_NO_SUMMARY_LINK: "1",
    DEPOT_NO_UPDATE_NOTIFIER: "1",
  },
});

depot() (from @depot/cli) runs the binary through execa, whose default is
extendEnv: true. So that env object is merged onto the parent process.env
rather than replacing it — the depot CLI inherits every variable trigger deploy has, including OTEL_EXPORTER_OTLP_ENDPOINT. The depot CLI then reads it
for its own OTel export and dies during telemetry init (empty gRPC target for a
malformed value, resource Schema-URL conflict for a valid one) before it builds
anything.

The value gets into the environment easily. A common case is a .env file: the
CLI loads it with plain dotenv, so whatever is in it ends up in the
environment the depot child inherits. (Ours is encrypted with dotenvx, so the
value the CLI sees is the literal encrypted:… string — a non-URL — which is
why we hit the empty target variant rather than the Schema-URL one.) Setting
the variable directly in the shell does the same thing.

Reproduction repo

https://github.com/cloudbring/trigger-depot-otel-endpoint-repro

To reproduce

From the repo above (trivial single task, nothing project-specific):

npm install
npx [email protected] login
export TRIGGER_PROJECT_REF=proj_xxxxxxxx

# fails — .env sets OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
npm run deploy

# succeeds — same project, same code, variable removed
mv .env .env.off && npm run deploy

Or without the file:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 npx [email protected] deploy   # fails
npx [email protected] deploy                                                     # succeeds
Additional information

--local-build is a working workaround, and so is unsetting
OTEL_EXPORTER_OTLP_ENDPOINT for the deploy — the latter keeps you on remote
builds. The local path runs docker buildx (x("docker", ["buildx", …]) in the
same file) which inherits the same variable but tolerates it, so the hard
failure is specific to the depot CLI's telemetry init.

Since some ambient env (PATH, HOME, …) is needed for depot to run, dropping
extendEnv wholesale probably isn't the fix. Options that would work: strip or
neutralize the OTel exporter variables for the depot child (e.g. delete
OTEL_EXPORTER_OTLP_ENDPOINT / set OTEL_SDK_DISABLED=true in that env), or
have the depot CLI not fail the build when its own telemetry endpoint is bad.
Happy to open a PR for the CLI-side version if that's the direction you'd want.

主要语言
TypeScript
星标
16.5k
派生
1.5k
平均合并
1 天 15 小时
30 天内合并 PR
17

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南,环境需要你自己搭建:先看它的 README,通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

triggerdotdev/trigger.dev 的其他 Issue

查看 triggerdotdev/trigger.dev 的全部 Issue

相似的 Issue

更多 TypeScript Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。