A month ago, I wrote a blog post explaining a hacky way to enable tree-shaking in Rails/Webpacker project at Simpl. I would definitely recommend skimming through the previous post if you have not already.
In this post, we will directly jump into a more robust and stable solution. But before that, I want to resurrect my old memories for you that haunted me for months wherein a broken manifest.json
was generated during webpack compilation at a random place. This time, after upgrading @rails/webpacker
and related webpack plugins, the problem has been escalated beyond repair wherein an incomplete but valid manifest.json
was generated randomly having fewer pack entries than expected. So even the generated manifest.json
has little chance of succor by the hacky NodeJS fix_manifest.js
script I had written to fix the broken JSON last time.
After a bit of googling my way out, I learned that webpack, with multi-compiler configurations, compiles each webpack configuration asynchronously and disorderly. Which is why I was getting an invalid manifest.json
earlier.
Imagine two webpack compilations running simultaneously and writing to the same manifest.json
at the same time:
{
"b.js": "/packs/b-b8a5b1d3c0c842052d48.js",
"b.js.map": "/packs/b-b8a5b1d3c0c842052d48.js.map"
} "a.js": "/packs/a-a3ea1bc1eb2b3544520a.js",
"a.js.map": "/packs/a-a3ea1bc1eb2b3544520a.js.map"
}
Using different manifest file for each pack
Yes, this is the robust and stable solution I came up with. First, you have to override Manifest fileName
in every webpack configuration in order to generate a separate Manifest file for each pack such as manifest-0.json
, manifest-1.json
, and so on. Then, use the same NodeJS script fix_manifest.js
with a slight modification to concatenate all the generated files into a final manifest.json
which will be accurate (having all the desired entries) and valid (JSON).
For that, we have to modify the existing generateMultiWebpackConfig
method (in ./config/webpack/environment.js
) in order to remove the existing clutter of disabling/enabling writeToEmit
flag in Manifest which we no longer need. Instead, we will create a deep copy of the original webpack configuration and override the Manifest plugin opts
for each entry. The deep copying is mandatory so that a unique Manifest fileName
can endure for each pack file.
const { environment } = require('@rails/webpacker')
const cloneDeep = require('lodash.clonedeep')
environment.generateMultiWebpackConfig = function(env) {
let webpackConfig = env.toWebpackConfig()
// extract entries to map later in order to generate separate
// webpack configuration for each entry.
// P.S. extremely important step for tree-shaking
let entries = Object.keys(webpackConfig.entry)
// Finally, map over extracted entries to generate a deep copy of
// Webpack configuration for each entry to override Manifest fileName
return entries.map((entryName, i) => {
let deepClonedConfig = cloneDeep(webpackConfig)
deepClonedConfig.plugins.forEach((plugin, j) => {
// A check for Manifest Plugin
if (plugin.opts && plugin.opts.fileName) {
deepClonedConfig.plugins[j].opts.fileName = `manifest-${i}.json`
}
})
return Object.assign(
{},
deepClonedConfig,
{ entry: { [entryName] : webpackConfig.entry[entryName] } }
)
})
}
Finally, we will update the ./config/webpack/fix_manifest.js
NodeJS script to concatenate all the generated Manifest files into a single manifest.json
file.
const fs = require('fs')
let manifestJSON = {}
fs.readdirSync('./public/packs/')
.filter((fileName) => fileName.indexOf('manifest-') === 0)
.forEach(fileName => {
manifestJSON = Object.assign(
manifestJSON,
JSON.parse(fs.readFileSync(`./public/packs/${fileName}`, 'utf8'))
)
})
fs.writeFileSync('./public/packs/manifest.json', JSON.stringify(manifestJSON))
Wrap up
Please note that the compilation of a huge number of JS/TS entries takes a lot of time and CPU, hence it is recommended to use this approach only in a Production environment. Additionally, set max_old_space_size
to handle the out-of-memory issue for production compilation as per your need – using 8000MB i.e. 8GB in here.
$ node --max_old_space_size=8000 node_modules/.bin/webpack --config config/webpack/production.js
$ node config/webpack/fix_manifest.js
Always run those commands one after the other to generate fit and fine manifest.json
😙