Build artifacts caching support for non-relocatable binaries · yarnpkg/yarn#480

(16 comments) (0 reactions) (0 assignees)JavaScript (41,514 stars) (2,731 forks)batch import

cat-featurehelp wantedneeds-discussiontriaged

Description

The current caching mechanism assumes all binaries are relocatable. However that's not always the case. As an example, the ocaml compiler itself has hardcoded paths inside the binary.

The current problem today is that if we build ocaml in project A and use the cache in building project B, artifacts in project B will be pointing to project A, where it was originally build. This means if we remove projectA, B will stop working.

One possible solution to this problem is to always run build in a configurable directory, instead of node_modules in the current project. After build we can then copy the build artifacts back to the destination (either project A or B). This way, the non-relocatable artifacts will only depend on a directory where users are aware of.

To repro:

mkdir A; cd A
yarn add @opam-alpha/ocaml
mkdir B; cd B
yarn add @opam-alpha/ocaml # This should be built instantly from cache
rm -rf ../A
node_modules/ocaml/bin/ocaml # ocaml in project B won't work anymore

@bestander @jordwalke @dxu

Contributor guide

Tech stack: javascriptnodejs
Domain: build systemcli
Issue type: feature
Difficulty: 3
Estimated time: 1-2 days
Activity status: stale
Clarity: clear
Prerequisites: Familiarity with package managersUnderstanding of build caching concepts
Newbie friendliness: 35
Research direction: Examine the current caching implementation in Yarn's source code (likely in the `src` directory). Review the proposed solution of using a configurable build directory. Check for any existing work or discussion in the comments on this issue. Assess feasibility of implementing the solution in the 1.x line given it is frozen.