renovatebot/renovate

Support `node_modules` that live in source control

Open

#13926 opened on Jan 31, 2022

View on GitHub
 (6 comments) (1 reaction) (0 assignees)TypeScript (14,066 stars) (1,916 forks)batch import
help wantedmanager:npmpriority-4-low

Description

What would you like Renovate to be able to do?

Having node_modules checked into source control is not a very common pattern, however there is at least one use case where it simplifies life a lot: creating Github Actions. GH expects all the things needed for runtime to be in the repository, so as a developer you're left with either checking in the node_modules or using a bundler.

The feature request is two-fold:

  1. The node_modules need to be wiped before renovation:
    • Without wiping, when used with a lockfile1, npm does not have the information about installed package meta data in the cache and so when it tries to recreate the shrinkwrap it does not have the checksums2. See an example resulting lock file maintenance PR: https://github.com/pkgjs/action/pull/31.
    • Moreover, when the node_modules are present, npm will not install the latest versions of the packages if the existing ones match the version range (unless npm update is run), so this effectively disables the core purpose of renovate.
  2. Need to push the updated node_modules back into source control after the upgrade.

If you have any ideas on how this should be implemented, please tell us here.

There's probably several considerations for this feature:

  • Does this need a separate config option or can it be implied (based on the presence of node_modules)?
  • Does this need separate config options for both of the steps - wiping node_modules at the start and adding them back at the end?
  • When lock files are not used, does this need a separate "node_modules maintenance" (which should probably run on a similar schedule as lock file maintenance?)

Alternatively, this could maybe become a little bit more abstract, not tied to node_modules, and just have separate config options for "wipe these files/folders at the start" and "check these folders/files into source control when done".

Is this a feature you are interested in implementing yourself?

Yes

Footnotes

  1. lock files are not strictly necessary when checking node_modules in, but it is one way to ensure that the whole dependency tree receives updates via renovate. ↩

  2. one could argue that this is an issue with npm, however we have to keep in mind, that if npm did not fetch and unpack the files, then it can't really know or calculate the checksum of the unpacked files, therefore leaving it out of the lockfiles is quite possibly the correct behavior. ↩

Contributor guide

Tech stack
typescriptnodejs
Domain
devopsci cdtooling
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Node.jsnpmGitRenovate internals
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
Review the issue comments for discussion on implementation approaches, particularly around wiping and re adding node modules. Examine the Renovate repository's npm manager code to understand current handling of node modules and lock files. Consider creating a prototype with separate config options for pre and post update scripts.