Is it necessary for a build server to remove node_modules before an AOT build?
I am currently dealing with an Angular application that is being deployed using an CI orchestrator and Jenkins.
Jenkins job is configured to do the following (relevant steps only):
- fetch sources from Git
- remove node_modules
- perform AOT build (
--prod+ other optimizations)
I have noticed that node_modules removal +
npm install + AOT build takes way more time than simply calling
npm install + perform AOT build, so I am wondering why the removal.
I have asked a few colleagues about this configuration and no one seems to know why the removal is required.
From what I know, removing node_modules is very rarely required (maybe some major update messes up some packages or similar) and I haven't removed any node_modules for any project in years (development environment).
So, is it necessary for a build server to remove node_modules before an AOT build?
I suspect this is an outdated practice:
Prior to npm 3, npm did not keep track of resolved dependencies, and npm install would try to reconcile the existing with the declared dependencies. Since
node_modules is not commonly under version control, this meant that the build would depend on hidden state, and therefore be non-reproducible. Back then, the easiest and most reliable way to ensure reproducible builds was deleting
Since npm 3, npm keeps track of resolved dependencies in
package-lock.json, thereby guaranteeing that the same dependency versions are used irrespective of the prior state of
node_modules (this should even work if the registry is updated retroactively, which sometimes happens to fix high priority security issues).
Nowadays, the only benefit of deleting
node_modules would be to guard against software other than npm tampering with its contents - but nobody should do that (and if somebody has hacked your build server, you probably have bigger problems ...)
1 comment thread
I can't really think of a compelling reason to remove
node_modules as a matter of course.
The most compelling one, is what you alluded to. If
node_modules is "corrupted" in some manner, removing and refetching it will resolve that "corruption" without needing any manual intervention. Short of a malicious attack, it's unlikely that you will fail to notice such "corruption" unless it really doesn't matter (e.g. corrupts code that you don't use). Still, if this was a concern, calculating a secure hash of
node_modules and verifying it would be a cheaper (though still fairly expensive) and more reliable method to check this (it's immune from upstream attacks assuming you trust the original versions you hash).
npm install does not verify the integrity of the
node_modules directory. It only verifies the integrity of the tarballs it downloads.
An important point to note is that
npm install will remove packages that are not in the
npm-shrinkwrap.json. Packages from earlier builds or otherwise present that are not explicitly depended upon will thus be removed avoiding depending on undeclared dependencies. This also avoids the need to manually clean up no longer necessary packages.
I would say assuming there are no costs and particularly that no one is bottle-necked on waiting for builds to complete, then endeavoring to ensure a consistent and clean build environment is typically preferable. It's certainly the safer, if more expensive, default.
0 comment threads