SystemJS to Webpack – Before You Begin

SystemJS to Webpack - Before You Begin Title

This is a primer discussing why to move from SystemJS to webpack in your Angular project. The post also describes some of the hurdles you may run into with this effort.

While it doesn't go into specific webpack configuration, this article aims to provide an overview for someone who has heard of webpack but doesn't quite understand why or how to get started. Specific details configuring webpack with ASP.NET Core are coming in future posts.

SystemJS – Starting Point

There are several posts on this blog with examples using SystemJS to mimic Node.js module resolution in the browser. While TypeScript understands how to resolve types by npm package name during development, SystemJS fills this need in the browser by mapping package names to specific script files within the npm packages. This is an effective and minimal way to setup an Angular 2 application to load its dependencies from npm and get going quickly.

After starting with SystemJS, applications (even small ones) quickly grow to contain hundreds of ES2015 modules that are then downloaded by the browser to execute the application. This is probably not a problem during development when the browser and the server are on the same machine.

At some point, however, you may want to host your application for others to see. The performance hit of loading this many modules becomes noticeable. This situation leads to creating some type of build combining these hundreds of requests into only a handful.

Whereas SystemJS manipulates the browser to understand node module resolution, webpack (and similar build tools) manipulate the code to accommodate the browser's needs. Whether SystemJS or webpack, you continue to code the way that makes you the most productive with as many ES2015 modules in as many files as make you happy. Unlike SystemJS, webpack provides the browser with a streamlined payload to load the application as efficiently as possible.

Destination Webpack

Webpack at its core is a JavaScript module bundler meaning it takes JavaScript modules from separate files and combines them into one file. Webpack can be extended, though, by using loaders which allow you to bundle other types of files. So, while webpack doesn't understand TypeScript natively, it processes .ts files by using a loader.

Webpack has another extensibility mechanism called plugins. Where loaders handle specific file types, plugins process the contents of the loaded files. Plugins, for example, are responsible for minifying code where variable names are shortened and whitespace is removed.

One aspect of webpack that is different from a tool such as grunt or gulp is that it is more declarative meaning you configure an object literal and that object defines what should happen when you run the webpack command. The declarative nature of webpack ideally requires less configuration code and should lead you into a build that is more optimized than if you had to wire the pieces up yourself.

There are two other npm packages that carry the webpack name but are not the bundler, itself. These packages work with the bundler to improve the development experience. They are webpack-dev-server and webpack-dev-middleware.

The webpack-dev-server npm package is a lightweight development server based on Node.js and Express. It has all the basic development server features and includes hot module replacement. Hot module replacement (HMR) is a process where the development server uses a WebSocket connection to replace the code of a given module in the browser when it has changed in the development environment. This does not require a full page refresh to reflect the updates making it seamless to view your changes in the browser during development.

The webpack-dev-middleware npm package is a little more esoteric. It runs as an in-memory store containing the build output as opposed to writing the output to disk. This can decrease the time between when you modify a file and when you can see the changes running in the browser.

Configuration Troubles

Currently in early 2017, webpack is moving from version 1 to version 2. This can lead to incompatibilities in the webpack tool chain. When configuring a webpack build yourself, expect to find out-of-date examples and see errors processing your build.

Not all loaders and plugins support both webpack 1 or webpack 2 and you may find yourself trying different versions of these npm packages with different versions of webpack. Throw in the fact that TypeScript is adding significant new features through all of this and it's a recipe for configuration troubles.

This isn't meant to scare you off, just be prepared. Expect to search through the GitHub issues of some of these frameworks looking for answers or expect to post issues yourself. It's not always clear which combination of settings are correct but the webpack community is by-and-large responsive and committed to this tooling.

This is an example of an issue you may find. While the TypeScript compiler allows comments in the tsconfig.json file as of version 1.8, some of the TypeScript webpack loaders will blow up if there are comments in this file. There is no helpful error message, it just throws an error that a property is not defined on a null reference.

It's likely these issues are resolved over time, but you should understand what you are up against.

Replacing SystemJS

Now that you know about webpack's advantages and potential pitfalls, what are the general steps required to replace SystemJS? This is a summary:

  • Move polyfill scripts from index.html into the webpack bundle
  • Compile and bundle the TypeScript files
  • Move any external component HTML templates and stylesheets inline
  • Add a reference to index.html for the new bundled resource

Move polyfill scripts from index.html into the webpack bundle

Many Angular tutorials instruct you to put polyfill dependencies directly in the index.html (or alternatively named default HTML) file. This approach works and will continue to work with webpack but you should consider moving these dependencies into the webpack configuration.

First, including these dependencies into a bundled request improves performance. Second, having all your script dependencies in one place decreases the time to find a given dependency – thus increasing maintainability. Third, webpack resolves dependencies through node module resolution meaning you add the package name to the configuration and you're done. Using SystemJS, you often must identify the correct script to load from within the npm package.

Compile and bundle the TypeScript files

While you could continue to compile TypeScript with the TypeScript compiler, the idea is to move everything into webpack. There are several loaders that handle TypeScript files and are generally straightforward to configure. A popular TypeScript loader in the Angular community now appears to be awesome-typescript-loader.

Move external component HTML templates and stylesheets inline

This one is Angular-specific. When you have components with external templates and/or stylesheets, Angular loads these files as separate requests. By using a loader like the angular2-template-loader, the build now can combine the contents of the external templates and stylesheets into the component script itself.

Note: Understand that by moving the templates and stylesheets inline, setting the @Component directive's moduleId property is unnecessary. Leaving the moduleId property set in the component can lead to issues. While the available tooling should most likely handle this, you may find you must account for this yourself by removing the settings manually or creating automation.

Add a reference to index.html for the new bundled resource

Finally, once you define the file(s) that webpack outputs, you must reference the new bundled file(s) in your main HTML file. As you may have guessed, there are even webpack plugins to handle this as well based off the webpack configuration (see html-webpack-plugin).

Additional Configuration

While the previous sections listed the necessary pieces of a basic webpack build. You have so many more options available to you to optimize your code. Some build steps to consider are: Ahead-of-time (AOT) compilation, using the wwwroot folder in ASP.NET Core, tree-shaking, running tests, and using the webpack development server with hot module replacement.

Tackling the Beast

So, you still want to get going with webpack? Where do you begin? There are many projects on Github and other sites that provide insight into how this tool works. It appears to still be hit or miss as far as quality. Looking at it from an ASP.NET developer's perspective, the guidance options are fewer. With that said, here are some links to get started:

None of these resources paint the full picture but they provide a good overview. In addition to these general resources, visit the documentation for the individual loaders and plugins to get details regarding compatibility with other packages.

You should consider creating your own webpack build from scratch. There are advantages to doing it this way. If you like knowing how these tools work, this is a rewarding way to learn, albeit a time-intensive one.

Focus on one piece of the build at a time. It's easier to identify which pieces of the build are failing and you can more easily investigate or replace that specific loader or plugin to compare results. Once you stabilize one part, move on to the next until you have the build that you want.

Start with the pieces outlined in the Replacing SystemJS section and continue from there.

More on the Way

You've learned why you should consider moving to webpack and the general approach to creating a webpack-based build. Again, this is just a primer. The plan is to ultimately provide you with a webpack configuration that works well with Angular and with ASP.NET Core. You will see more posts about webpack here soon.

For now, what are your favorite webpack resources? Did you build a template that others may find useful? Please share in the comments.

Get Your TypeScript Definitions from NPM

Get Your TypeScript Definitions from npm

Derived from photo by raumrot.com, CC-BY

If you use TypeScript, you appreciate what the type system provides with IntelliSense and easier refactoring. This experience is quite seemless with code you have written. However, using third-party libraries has left many struggling with complex configurations.

With the advent of ECMAScript 2015 (ES2015) modules and the rise of npm as the defacto JavaScript package manager, configuring your type definitions has gotten more complicated. Not only did you have to ensure that your import statements resolved to the correct scripts but now also the correct type definitions.

Thankfully, the community is standardizing third-party type resolution similar to how it is standardizing on ES2015 modules and npm.

Once Upon a Time

Back in the early days of TypeScript, you used to reference type definitions at the top of the .ts file like this:

/// <reference path="../lib/my-favorite-library.d.ts" />

Also you probably got the type definitions from a dedicated registry somewhere like DefinitelyTyped and maybe even used command line tools to setup and configure these type definitions like tsd or typings.

There were shortcuts and optimizations but ultimately it was a whole other set of configuration and package management to get third party libraries compiling with TypeScript.

Relative Path Module Resolution

While you can still write your comment-based type references, the industry has adopted the ES2015 module standard and so has TypeScript.

By adopting this standard, each JavaScript (or TypeScript) file essentially represents a module with its own scope. Any dependencies that this module requires are imported directly from another module by the module's URL. In the future, browsers will understand how to parse this syntax and load each module as needed from the server. Today, TypeScript transpiles these import statements into a configurable module loader syntax such as AMD, UMD, or CommonJS.

Take for example the fact that your GrooveService takes a dependency on your FunkService. Because you wrote your FunkService in TypeScript, both the scripts and the types are found at the same location.

import { FunkService } from './funk-service';

On the other hand, what if you are using angular in Angular 1.x? Initially, you might think to reference it by relative path:

import * as angular from './node_modules/angular/index.js';

This makes sense because this location is where the scripts reside and the browser will load the correct library. However, TypeScript complains that it can't find angular. Well, it turns out the angular library doesn't include TypeScript definitions in the package. TypeScript tried to find the type definitions at the relative path, but they weren't there and you got the dreaded red squiggle..

Angular IntelliSense Squiggle

Traditionally, this is where tsd or typings came into the picture. These repositories contained packaged type defintions for many popular libraries independent of the library provider. And while these repositories solved a real problem in obtaining type definitions, they weren't easy to setup and configure. After all, who wants to maintain two sets of packages managed by two different tools with two different configurations just for the ability to reference one library using TypeScript?

There's another problem with taking a relative path reference to an npm module, what if your code is distributed as an npm package? If you have spent time reading up on npm dependency resolution, you know that you can't guarantee the location of your dependencies in npm. They may be dropped at different levels of the folder structure and therefore attempts to reference npm modules by relative path are fragile.

Node Module Resolution

In order to solve the npm module resolution problem, the folks at TypeScript decided to just follow the npm model. This solution makes it seamless to reference both the code and the types using the same syntax used in NodeJS.

import * as angular from 'angular';

Given this import statement, NodeJS understands that angular is equivalent to /node_modules/**/angular/index.js. It doesn't know exactly which directory it will be in but it knows how to find it.

As of TypeScript 1.6, the compiler understands how to look for type definitions in the same fashion. TypeScript looks for something roughly equivalent to /node_modules/**/angular/index.d.ts. This is great because it ensures that the code and the types can be found in the same npm package.

But what if the code and the types aren't in the same package? For instance, the Angular 2 libraries ship with type definitions included in the package but Angular 1.x does not. This introduces another great solution from the community. Instead of using tsd or typings for your third-party type defintions, now you can just use npm.

The @types organization is available on the public npm registry for the purpose of hosting third-party type definitions that don't ship with their repective package. So if there is a set of types available independent of the package's publisher, by convention you can find them on npm using @types/package-name. For example to download types for angular, you get them by installing the @types/angular npm package. Furthermore, TypeScript understands how to resolve types from the @types directory out of the box.

To ensure node module resolution for your project, set it in your tsconfig.json file:

{
    "compilerOptions": {        
        "moduleResolution": "node"
    }
}

Note: To read more about how TypeScript handles node module resolution, there are more details in the TypeScript documentation.

Bringing Node Module Resolution to the Browser

This all sounds great – less tooling dependencies, easier configuration. NodeJS runs everywhere – desktop clients, web servers, Linux, MacOS, Windows. But unfortunately, when the web browser tries to load scripts from angular you get a 404 result. The browser doesn't understand how to convert the npm package name into the URL for which the script resides.

This is the place where the node module abstraction breaks down. But think about it for a second. Yes, NodeJS doesn't run in the browser. Yes, you aren't necessarily writing a NodeJS backend. But what NodeJS has done is unify the runtime on which web developers run their client-side tooling. This is the environment on which you do your TypeScript compilation, combining modules, minifying – basically optimizing your code to be consumed by the browser. And so it still makes sense that npm would be the best solution for module resolution of both the scripts and the types.

There are a ton of different ways to prepare your application's npm modules for delivery to the browser and you can bet that there will be more coming in the future. For now, SystemJS is a good place to start. Many deveopers already use this library as a script loader and by adding some additional configuration, it's possible to map npm-based dependencies to a relative URL.

Without any additional configuration, the browser is going to see the import statment for angular and attempt to request /angular from the server. What should really happen is that the browser requests the code from the actual package folder like node_modules/angular/index.js. So using your SystemJS configuration file, add the following:

System.config({        
    packages: {
        '': {
            defaultExtension: 'js',
            map: {
                'angular': './node_modules/angular/index.js'
            }
        }
    }
});

This code configures the root of your application (at the path '') to always add a .js extension if one is not present and to also map the 'angular' path. Now when SystemJS sees the import statement for 'angular', it maps that to a request for node_modules/angular/index.js as defined in the configuration.

Automatic Typings Acquisition

If you've made it this far, you understand how this simplifies the process of acquiring and referencing type definitions. It's become simple enough in fact that the Visual Studio Code team is working to make it automatic in the editor. As of this writing, the feature is released but the team admits it still has rough edges. Even still, you can read more about it in the release notes and start trying it by downloading version 1.7.2 or later.

Keep in mind, the tooling may have drawbacks even when it is complete and it's helpful to know what the tooling is doing underneath the covers. Also, this tooling doesn't address the need for mapping npm package names to URLs in the browser like discussed with SystemJS.

The End

The point here is that node module resolution has become the defacto standard in the web development community and TypeScript is no exception. Now that @types is available on npm, you have one place to get your third-party libraries and type definitions – npm.

The tricky party is still configuring node module resolution for the browser but you see how it's possible to setup SystemJS to handle the mapping. Look for more tooling discussion in future posts.

Are you using ES2015 modules and node module resolution with TypeScript? What is your preferred method of mapping the module names to URLs for the browser?

Using EcmaScript 2015 Modules in TypeScript with SystemJS

TypeScript and SystemJS header

There are two powerful tools for Angular developers in TypeScript and SystemJS. They are both in the Angular 2 quick start and they work just as well with AngularJS 1.x for building maintainable Angular applications.

However, getting started using these tools can be tricky. You need to align both tools' configurations so they work together and having a basic understanding of what happens under the hood can save you hours of confusion and frustration. Once you understand the core concepts and setup the configuration, this workflow helps to make your code more decoupled and maintainable – and maybe even more fun.

This post walks through the basics of using SystemJS and TypeScript in Visual Studio for development. You learn the minimum configuration that matters, how it interacts with the rest of the system, and what to expect when compiling and running your application.

Before You Begin

If you want to skip right to the code, it is all on GitHub. The example is in Visual Studio 2015 (there's a free community edition) and is designed to run with minimal dependencies.

The example project is a component called ListMaker which allows the user to make a list by typing in a textbox and pressing ENTER. This component is made up of two child components AddItemTextbox and List, and includes basic CSS styles. The main script bootstraps the ListMaker component and everything is brought together in index.html.

Application structure

SystemJS is the only library used in the example. Adding dependencies with TypeScript and SystemJS, including Angular, is another level of complexity that is omitted here intentionally. There are enough pieces to understand using SystemJS and TypeScript together so watch for more posts on this topic.

Required Tools and Frameworks

This walkthrough uses the following tools and frameworks:

Visual Studio

The demo is built using Visual Studio 2015 Community. Visual Studio provides a seamless experience to compile and run your application on a development server. It also has first class support for TypeScript.

TypeScript Add-In for Visual Studio

This add-in enables TypeScript support in Visual Studio. The demo uses TypeScript 1.8.36. Install the add-in through Visual Studio's Extensions and Updates manager or from the web.

SystemJS

SystemJS is the JavaScript module loader. It's included in the project on GitHub. You can also obtain it from your favorite JavaScript package manager or as a release download from GitHub.

If you don't use Visual Studio, don't fret. You can port the code to another web project as there are no Visual Studio-specific dependencies. Be sure you know how to setup a development server and TypeScript support in your environment of choice.

Workflow Overview

When using these tools together, your workflow requires a compilation step. The Visual Studio TypeScript add-in automatically handles the compilation when you build or run your project. If you aren't in the habit of using front-end compilation tools such as TypeScript, this may be new to you. Here is an overview:

Diagram of TypeScript with SystemJS workflow

TypeScript Configuration

While the default TypeScript configuration in Visual Studio works, there are a couple options to discuss. First, you should consider adding a tsconfig.json file to your project. By default, you find the TypeScript properties in the Visual Studio project properties, however using a tsconfig.json file has advantages.

Most importantly, the project properties do not show all of the options that are available in tsconfig.json so if at some point you want to set other options, you will be limited by the tooling in the project properties. Also, the tsconfig.json file is a standard configuration for every editor and compiler that supports TypeScript. Therfore if someone joins your team that knows TypeScript, they will know where to look for the configuration no matter which tools they use.

Note: you should install the latest update of Visual Studio 2015. Earlier versions did not properly recognize the tsconfig.json file.

The first related configuration option is compilerOption.module. This option sets which module loader format the TypeScript compiler emits. Because SystemJS understands all of them, it's hard to pick the wrong one. Ultimately your choice here may depend on the other tooling you use for building your JavaScript. For now, choose "system" as it is the native format of the loader but you may find that "amd" provides greater compatibility with other compilation tools.

The other option is compilerOption.target. This option sets which version of the ECMAScript® standard to target when generating JavaScript from your TypeScript files. This value largely depends on which browsers you are targeting. The default value is "es3" which gives you the broadest range of browser support. It's worth noting that should you eventually want to emit native ECMAScript module syntax, this option will need to be set to "es2015".

Your tsconfig.json file should at least have:

Implement ECMAScript 2015 Module Syntax

Take a look at the ListMaker component without the ECMAScript 2015 module syntax:

The first thing to note is that ListMaker is in the global scope as it is meant to be re-used within a given application. The ListMaker implementation, however, is not in global scope and is instead wrapped in an immediately invoked function expression (IIFE). If you look closely, you also see that ListMaker depends on two other constructor functions not defined in this code: AddItemTextBox and List. These constructors are also defined in global scope and furthermore you have to ensure that AddItemTextBox and List are loaded in the browser before trying to create a ListMaker.

The fundamental mechanism for modules are import and export statements. The import statements define the module's dependencies and the export statements define the objects that are available to other modules. Take a look at how the ListMaker code changes when using ECMAScript modules:

The module's dependencies are clearly defined in the import statment and you can easily determine where to obtain the source code for these objects looking at its URL. Also, because module scope is separated from global scope by default, the use of an IIFE is unnecessary leading to less code and more readable code.

Finally notice the export statement and the use of the default keyword. While ECMAScript modules can define multiple exports, only one object is defined as the default export. Similarly, the import statements for this module are explicitly importing the default export from the target modules.

This post doesn't go into all the ways you can write your import and export statements. TypeScript supports various options for writing these definitions so read more about them here.

SystemJS Configuration and Reconciling Import URLs

This is likely the most important part configuring SystemJS and TypeScript to work together. Both SystemJS and TypeScript evaluate the import statements. When TypeScript looks at the import statement, it is looking for a TypeScript file so that the compiler can use the types defined in that file. Using import './Foo', TypeScript looks in the current directory for Foo.ts and then uses the information contained in that file for compilation and type checking.

SystemJS on the other hand looks at import './Foo' and then attempts to make a request to the server for <current-directory>/Foo and without any special configuration, this request will produce a 404 resource not found error. What SystemJS needs to request instead is <current-directory>/Foo.js.

Luckily, SystemJS has a configuration option to append a file extension to requests. You configure this by setting up a specific path in your application using the packages option. In the demo, all of the TypeScript files are in the components folder and SystemJS needs to add a .js extension to these requests.

Create a systemjs.config.js file in your application and add the following configuration:

You can optionally configure this behavior for the whole application by using an empty string '' to set the root of your application as the path. Also remember this configuration only applies to requests made by SystemJS and other requests still require explicit file extensions. There are many useful options in the SystemJS configuration so be sure to read up on its capabilities.

Bootstrap the Application

There is bootstrapping logic for the application in main.ts. The browser requests this file first and the script's purpose is to access the Document Object Model (DOM) and render the root component which in this case is ListMaker. It's important to understand using module loaders that the first module retrieved is the last one to execute its contents. For example, even though main.ts is the entry point of the application, all of the other dependent scripts execute prior to main.ts executing.

Once you determine the main script, you bring the configuration and bootstrapping together on the root HTML page. Reference SystemJS, followed by the SystemJS configuration, and then use the System.import syntax from SystemJS to load the post-compiled version of main, 'scripts/main'. Remember that even though the extension is not present, you have configured SystemJS to add the extension and request scripts/main.js.

Once main.js is loaded, all of the other import statements are evaluated and the target modules are loaded as well to compose the application. This diagram illustrates the dependencies and the order of execution when loading.

Diagram illustrating bootstrap process

At this point, you have configured enough for your application to run. Remember to compile the TypeScript first and all of the modules will run and execute in the correct order. Be sure to check out the complete project for all of the details.

Only the Beginning

This is the first step using ECMAScript 2015 modules with SystemJS and TypeScript and this workflow works quite well as a development workflow. Once you understand the configuration and basics of using the tools, you can get up and running rather quickly.

But you need more. As an Angular developer, you need to bring in third-party libraries and seamlessly integrate with your application code. This is the topic of the next post in this series.

At some point, you will want to bundle these scripts together into one request. You may even want to create several bundles that are loaded dynamically via a route change or a certain screen size. The good news is that by separating your code into modules that have explicit dependency references, you have made this configuration much easier to attain.

Finally, the documentation for both of these tools is well done. Be sure to read the TypeScript documentation and the SystemJS documentation for more information.

Stay tuned for more on this topic. If there is anything else you want to know about these tools or if you also have experience with them, please share in the comments below.