Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics

Dashboard
Notifications
Mark all as read
Q&A

Is it necessary for a build server to remove node_modules before an AOT build?

+1
−0

I am currently dealing with an Angular application that is being deployed using an CI orchestrator and Jenkins.

Jenkins job is configured to do the following (relevant steps only):

  • fetch sources from Git
  • remove node_modules
  • npm install
  • perform AOT build (--prod + other optimizations)
  • deploy

I have noticed that node_modules removal + npm install + AOT build takes way more time than simply calling npm install + perform AOT build, so I am wondering why the removal.

I have asked a few colleagues about this configuration and no one seems to know why the removal is required.

From what I know, removing node_modules is very rarely required (maybe some major update messes up some packages or similar) and I haven't removed any node_modules for any project in years (development environment).

So, is it necessary for a build server to remove node_modules before an AOT build?

Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

0 comments

2 answers

+3
−0

I suspect this is an outdated practice:

Prior to npm 3, npm did not keep track of resolved dependencies, and npm install would try to reconcile the existing with the declared dependencies. Since node_modules is not commonly under version control, this meant that the build would depend on hidden state, and therefore be non-reproducible. Back then, the easiest and most reliable way to ensure reproducible builds was deleting node_modules.

Since npm 3, npm keeps track of resolved dependencies in package-lock.json, thereby guaranteeing that the same dependency versions are used irrespective of the prior state of node_modules (this should even work if the registry is updated retroactively, which sometimes happens to fix high priority security issues).

Nowadays, the only benefit of deleting node_modules would be to guard against software other than npm tampering with its contents - but nobody should do that (and if somebody has hacked your build server, you probably have bigger problems ...)

Why does this post require moderator attention?
You might want to add some details to your flag.

1 comment

I also suspect this reason. Unfortunately, where I work it sometimes happens that folks to just copy-paste old projects configuration (in this case Jenkins job configuration) without wondering why a step is there. Thanks. Alexei‭ about 1 month ago

+3
−0

I can't really think of a compelling reason to remove node_modules as a matter of course.

The most compelling one, is what you alluded to. If node_modules is "corrupted" in some manner, removing and refetching it will resolve that "corruption" without needing any manual intervention. Short of a malicious attack, it's unlikely that you will fail to notice such "corruption" unless it really doesn't matter (e.g. corrupts code that you don't use). Still, if this was a concern, calculating a secure hash of node_modules and verifying it would be a cheaper (though still fairly expensive) and more reliable method to check this (it's immune from upstream attacks assuming you trust the original versions you hash). npm install does not verify the integrity of the node_modules directory. It only verifies the integrity of the tarballs it downloads.

An important point to note is that npm install will remove packages that are not in the package.json/package-lock.json/npm-shrinkwrap.json. Packages from earlier builds or otherwise present that are not explicitly depended upon will thus be removed avoiding depending on undeclared dependencies. This also avoids the need to manually clean up no longer necessary packages.

I would say assuming there are no costs and particularly that no one is bottle-necked on waiting for builds to complete, then endeavoring to ensure a consistent and clean build environment is typically preferable. It's certainly the safer, if more expensive, default.

Why does this post require moderator attention?
You might want to add some details to your flag.

0 comments

Sign up to answer this question »