Modular d3 app with commonJS and webpack

Creating modular d3 app with commonJS and webpack

In my previous post, I wrote about how to break up the d3 visualization app into different modules. It turns out that there is a better way to write modular d3 app, using commonJS and webpack.

Update on 6/May/2016:(webpack 1, deprecated) For a more idiomatic approach using webpack.optimize.CommonsChunkPlugin, visit webpack’s documentation on code splitting. The approach described in this post has the limitation of not being able to produce separate output files for app and vendor libraries, which is addressed by the idiomatic approach.

Update on 23/March/2017: (webpack 2, recommended) For a more idiomatic approach using CommonsChunkPlugin, visit webpack 2’s guide on code splitting. The approach described in this post has the limitation of not being able to produce separate output files for app and vendor libraries, which is addressed by the idiomatic approach.

Update on 12/Sep/2016: This post is gaining lots of popularity since d3.js v4 is released under npm. The approach described below should still work in theory although I have not tested it. Let me know if it doesn’t.

Writing in commonJS

Instead of using the revealing module pattern, a better way to modularize the JavaScript code is to actually adopt a standard. Currently there are two standards on writing modular JavaScript code, commonJS and AMD (Asynchronous Module Definition). I prefer to write in commonJS just because it is the same as the implementation in node.js which I am familiar with.

Note that commonJS is actually not a standard for browsers. It was designed to be used outside the browser.

So why are we using that? It is because currently the ES6 standard for modules are not supported in any browsers, and we need an alternative to write modular JavaScript code for the time being. Hence commonJS is a good alternative until the ES6 syntax becomes widely adopted.

All you need to do to write in commonJS is to use keywords module.exports  and require. For example, in my UI module for gitviz, I would write:

You can see the difference between commonJS and the revealing module pattern is very minor. So you can easily turn your codes into commonJS by adding  module.exports and removing the namespace identifier (since we will not expose any variables within a module to the global object).

Running commonJS in browser with webpack

We cannot run commonJS directly in the browser. In order to run commonJS in browsers, we need to:

• First, use a JavaScript builder/bundler that to turn source code written in commonJS syntax into ES5 syntax, which the browser recognizes.
• Second, load the generated JavaScript code in ES5 into the HTML file.

There are a few popular choices for builder/bundler:

• Rollup, which is mentioned in the official d3.js documentation
• Browserify
• webpack, newer than Browserify

I will use webpack in this post since I am more familiar with it.

One of the main functions of such as tool is to bundle the JavaScript codes written in plain JavaScript/commonJS/AMD together and handle any additional dependencies (such as d3, jQuery or any other JavaScript libraries) through commonJS or AMD.

For example, I have the following code in the source code for one of my modules:

Webpack will recognize this commonJS syntax and look for the dependency d3.tip in the directory ./vendor/d3-tip and bundle the library code together my other codes, making sure the dependencies are correctly met.

There are several ways to run webpack, my favorite way is to use a build system like gulp.js. You will need to specify a gulpfile.js , which will look something like this one I used in my project:

In this case we pipe the original source files in  public/js/ to webpack and output the bundled JavaScript code to  public/dist/ .

I used a webpack.ProvidePlugin (ProvidePlugin in webpack 2) to specify some modules as “global” so that they are automatically loaded for all other modules.

Notice that d3 is included as a module but it does not have a relative path. This is because some libraries like d3.js offer the library through npm, making it possible to load the module through node.js module system instead of using standalone libraries. All we need to do it to execute  npm install d3 --save and adds d3 as a node module, which will be recognized by webpack automatically.

The magic

After setting up webpack and gulp, the rest is just magic:

There you will see that you have a shiny  bundle.js file in your dist directory which you can throw into your front-end HTML file of your app:

The benefits

“Why all these?” You might ask the obvious question at this stage. I would say there are several reasons why I would prefer to write in commonJS over the old revealing module pattern:

• Better code organization. Your code becomes more organized and modular with all the explicit module dependencies instead of all the “undeclared” variables that are all over the place in different JavaScript files.
• Easy dependency management. You want to use a 3rd party standalone library? Just require it and let others know that you have that dependency. You want to a wonderful library but it is only available through npm? just  npm install and add it to the webpack.
• Tons of plugins. You can do a lot more when you use a bundler like webpack. If you decided to try out the es6 features but fear to break browser supports, you can use Babel as a webpack plugin to transform your code written in es6 back to es5 syntax for full support of all browsers (WOW). You can also uglify and minimize your code to save time in serving the JavaScript files, in addition to the time saved by bundling all the files together.

For an example of modular d3 app, you can check out my gitviz.