ASP.NET Core – Migrating to MSBuild

Migrating to MSBuild Header

This post continues the theme of exploring project structures for Angular and ASP.NET Core applications. The ASP.NET Core framework is in the midst of a transition to a new MSBuild-based project configuration. While there is material to help you start with MSBuild from scratch, many if not most of the published ASP.NET Core examples use the JSON-based project system. By learning how to migrate an existing application to the new MSBuild system, you can leverage the existing code and learn the new system – it's the best of both worlds.

There have been a couple high-profile posts recently around migrating from project.json to MSBuild from Nate McMaster and Scott Hanselman. This post is an overview of the MSBuild migration to help put these recent conversations in context. Why is Microsoft doing it? How can you migrate easily? The post covers these topics and includes links and tips to help with your own migration.

Moving to MSBuild

MSBuild, also known as the Microsoft Build Engine, is an XML-based project system for processing and building software. It's designed specifically for the .NET Framework and Visual Studio but is capable of processing other types of software and frameworks.

When .NET Core was introduced, ASP.NET Core was one of the first application targets available. To make the platform more attractive to open-source web developers, ASP.NET Core adopted a JSON-based project system defined in a project.json file. Looking at Node.js, everything is JSON-based as opposed to XML so this was a sensible design choice to attract new developers.

Prior to .NET Core, .NET projects were all based on MSBuild (or an MSBuild-like system). Typically, the developer didn't have to worry about this. Most projects started from built-in Visual Studio templates and Visual Studio automatically handled updates to project files (.csproj, .vbproj, etc.). Occassionally you had to resolve merge conflicts in the project file because it contained a list of each file in the project which was represented by its own XML node. If two developers independently modified the list of files in a project, this could result in a merge conflict but these issues were typically easy to resolve.

After .NET Core debuted, it quickly positioned itself as the framework for all .NET-based application targets including Universal Windows Platform (UWP) and Xamarin. Non-web developers didn't see the project.json file as an upgrade. They were used to using MSBuild and wanted to continue to use it. So for the greater good of the existing .NET community, Microsoft announced they are going to re-standardize on MSBuild as the build system for .NET Core.

Microsoft has added new capabilities to MSBuild. It went cross platform in 2015 and Microsoft announced enhancements for the build engine. Read more about them in Microsoft's blog post. The first MSBuild-based SDK 'alpha' was released in November, 2016.

New CLI Command, "dotnet migrate"

To aid in the transition to the MSBuild project system, the dotnet command-line interface (CLI) now includes the dotnet migrate command. This command takes a project using JSON-based configuration and converts it to a .csproj file. The command is available in dotnet CLI version 1.0.0-preview3-004056 or later.

The utility carries out several actions. First of all, it takes your existing project.json, global.json, and .xproj files and moves them to a folder named backup. If you are using source control, you can safely delete this folder.

The second thing the utility does is generate a new .csproj file from the contents of the project.json file. Before you delete the old file, consider taking a look at how the project.json and .csproj files compare.

Most of the contents of the project.json convert to PackageReference elements:

<PackageReference Include="Microsoft.AspNetCore.Mvc">
    <Version>1.1.0</Version>
</PackageReference>

There is a slight difference for tooling dependencies. They are added as DotNetCliToolReference elements.

<DotNetCliToolReference Include="Microsoft.DotNet.Watcher.Tools">
    <Version>1.0.0-msbuild1-final</Version>
</DotNetCliToolReference>

For executing npm scripts in your ASP.NET build to pre-process your TypeScript or Sass code, the new MSBuild script looks like this:

<Target Name="PrepublishScript" BeforeTargets="Publish">
    <Exec Command="npm install" />
    <Exec Command="npm run tsc" />
</Target>

If you are distraught over the project file moving from a concise JSON schema to a specialized XML-based schema like MSBuild, don't get too upset yet. First, you can continue to use project.json in the short-term by including a global.json file in your project. In the long term, Microsoft has stated they are looking at how to generate MSBuild-based configuration so that ideally you don't have to write it yourself. The dotnet migrate command is an important first step to this commitment. However, as an ASP.NET developer you will most-likely benefit from having a basic understanding of MSBuild.

Tips

There is a great post discussing the details of migrating from project.json to MSBuild in Nate McMaster's conversion guide. Also, take a look at the MSBuild documentation to gain a general understanding of the build system.

One important detail to note is that your global.json file must reference a version of the CLI that supports dotnet migrate (1.0.0-preview3-004056 or later). If you see the error No executable found matching command "dotnet-migrate", look to update the sdk in your global.json file.

This last tip is anecdotal. It did cost me time so hopefully describing it may help someone else. While not completely sure the reason yet, there was an issue executing commands, in this case npm scripts, before or after the "Publish" target. If you experience this issue, try using a different target such as Build.

<Target Name="NpmCommands" AfterTargets="Build">
    <Exec Command="npm install" />
    <Exec Command="npm run tsc" />
</Target>

Finally, look for more resources from Microsoft. They are working to deliver an "RTM quality" version of the SDK in early 2017 so expect many updates to documentation and templates leading up to the release.

Conclusion

The dotnet migrate tool is very helpful. I have found works well at least with smaller projects. This is a considerable shift in configuration and Microsoft has done commendable work to automate the process. Looking forward, it will be interesting to see how they make this configuration easier for web developers coming new to the platform.

Have you been working with the MSBuild project system in .NET Core? What have been your pitfalls or successes? Please share in the comments.

Great Angular, ASP.NET Core Starter Templates

Great Angular, ASP.NET Core Starter Templates Title

This post exists to provide hope. After the somewhat hesitant reaction to last week's webpack post, this post highlights two great Angular and ASP.NET Core starter templates. These starter projects not only contain configurations for webpack 1 and webpack 2, there are a ton of other Angular and ASP.NET features implemented for you.

ASP.NET Core Template Pack

ASP.NET Core Template Pack - Angular Starter

The first project template to note comes from the ASP.NET Core Template Pack by Mads Kristensen. The template pack is a Visual Studio Extension containing multiple templates for ASP.NET Core. The collection includes a helpful Angular template.

Key Features

The hallmark feature of the template is Angular Universal. This framework enables server-side Angular rendering when the application is initially requested from the server. Angular Universal decreases the initial page load time by rendering the first request on the server instead of requiring the client to wait for the application to download before rendering. Subsequent route changes are all rendered client-side. For any application that requires minimum load times, this framework is a must-see.

The template also uses webpack version 1. For anyone working on integrating webpack into their applications, every example helps. This template doesn't use the latest loaders that the folks on the Angular side of the community are using. It does however accomplish the primary goal by compiling combined scripts. And being that it's using Angular Universal, there are webpack configurations for both the client application and the server rendering portion.

The template also implements webpack's hot module replacement feature. This is a development feature where the client-side code is replaced in the browser via a WebSocket when the source module is edited on the file system. If you've ever used a development server with automatic refresh like Browser Link or Browsersync, then it's like that – except without the full page refresh. You should check out the template just to try out this feature – you may never go back.

The previously mentioned features are configured with the help of Steve Sanderson's ASP.NET Core JavaScript Services. This library is part of ASP.NET Core on GitHub and provides useful integrations for single-page application (SPA) frameworks including Angular. The template includes simple, foundational examples for routing and data access as well as Docker container configuration.

Gotchas

Currently, if you are looking for the very latest in front-end Angular features, this template doesn't have them. It's using webpack version 1 versus webpack version 2 and doesn't include all of the niceties that have been introduced with newer webpack loaders such as awesome-typecript-loader and angular2-template-loader.

Also, if you see an error like this when starting the template, it means you have to specify a prior version of ASP.NET using the global.json file.

Template Error

You can read more about the solution on GitHub.

Bottom Line

This is a stable template that highlights the main integration points between ASP.NET Core and Angular. For now, it might not have the most up-to-date client-side features but you can imagine these will come as the tooling comes to a final release (or when we all submit pull requests). Being a Visual Studio extension, it is optimized for the Visual Studio IDE and it provides a nice starting point.

To view the source for this template, check out its GitHub repository.

ASP.NET Core & Angular2 Universal Starter (Angular2Spa)

ASP.NET Core & Angular2 Universal Starter Diagram

This might be the best Angular on ASP.NET Core template available right now. It's actively developed by a member of the Angular Universal team, Mark Pieszak.

In fact during the course of writing this review, he fixed an issue that I was experiencing while working with the template for this article. The template contains some of the latest techniques and frameworks for building Angular applications on ASP.NET or otherwise.

Key Features

You can think of this project as a superset of the template currently in the ASP.NET Core Template Pack. It has the same base level of features and then adds on top of it. This template works in Visual Studio. It also contains configuration files to run in Visual Studio Code, complete with key bindings to initialize the dotnet build. It even uses the newer ASP.NET Core 1.1 assembly versions.

For the front-end tooling, the template uses webpack version 2 including many new loader patterns that you find in the Angular documentation. The example also makes use of an RxJS-based, Redux-like store via the NgRx library.

To help keep your code styles on par, this template uses codelyzer. Built on top of TSLint, codelyzer provides linting for TypeScript as well as the Angular Style Guide conventions.

Finally, this template configures unit testing with Karma and Jasmine integrated with webpack. It also includes end-to-end (E2E) tests using Protractor. These configurations alone can save you a dozen hours versus configuring yourself.

Gotchas

Certainly check out the README file for this project. It has a lot of great information on how to use Angular with or without this template and highlights areas where you may otherwise stumble. One other word of caution – this template leans on the bleeding edge so you may find some instability along the way. The maintainer so far seems to do a nice job keeping up with any logged issues.

Bottom Line

This is a must-see and must-watch starter template for ASP.NET developers using Angular. There is nothing more to say, take a look for yourself.

Final Thoughts

The Angular and ASP.NET frameworks are both evolving at a rapid pace and it can be difficult to keep current with the changes. These templates can help you start coding quickly and help you incorporate new technologies and patterns as they are published.

These are two of the best and more noteworthy starter templates for Angular and ASP.NET. Give them a try and share your thoughts below. If there are any other starter templates you find helpful, please let everyone know in the comments.

SystemJS to Webpack – Before You Begin

SystemJS to Webpack - Before You Begin Title

This is a primer discussing why to move from SystemJS to webpack in your Angular project. The post also describes some of the hurdles you may run into with this effort.

While it doesn't go into specific webpack configuration, this article aims to provide an overview for someone who has heard of webpack but doesn't quite understand why or how to get started. Specific details configuring webpack with ASP.NET Core are coming in future posts.

SystemJS – Starting Point

There are several posts on this blog with examples using SystemJS to mimic Node.js module resolution in the browser. While TypeScript understands how to resolve types by npm package name during development, SystemJS fills this need in the browser by mapping package names to specific script files within the npm packages. This is an effective and minimal way to setup an Angular 2 application to load its dependencies from npm and get going quickly.

After starting with SystemJS, applications (even small ones) quickly grow to contain hundreds of ES2015 modules that are then downloaded by the browser to execute the application. This is probably not a problem during development when the browser and the server are on the same machine.

At some point, however, you may want to host your application for others to see. The performance hit of loading this many modules becomes noticeable. This situation leads to creating some type of build combining these hundreds of requests into only a handful.

Whereas SystemJS manipulates the browser to understand node module resolution, webpack (and similar build tools) manipulate the code to accommodate the browser's needs. Whether SystemJS or webpack, you continue to code the way that makes you the most productive with as many ES2015 modules in as many files as make you happy. Unlike SystemJS, webpack provides the browser with a streamlined payload to load the application as efficiently as possible.

Destination Webpack

Webpack at its core is a JavaScript module bundler meaning it takes JavaScript modules from separate files and combines them into one file. Webpack can be extended, though, by using loaders which allow you to bundle other types of files. So, while webpack doesn't understand TypeScript natively, it processes .ts files by using a loader.

Webpack has another extensibility mechanism called plugins. Where loaders handle specific file types, plugins process the contents of the loaded files. Plugins, for example, are responsible for minifying code where variable names are shortened and whitespace is removed.

One aspect of webpack that is different from a tool such as grunt or gulp is that it is more declarative meaning you configure an object literal and that object defines what should happen when you run the webpack command. The declarative nature of webpack ideally requires less configuration code and should lead you into a build that is more optimized than if you had to wire the pieces up yourself.

There are two other npm packages that carry the webpack name but are not the bundler, itself. These packages work with the bundler to improve the development experience. They are webpack-dev-server and webpack-dev-middleware.

The webpack-dev-server npm package is a lightweight development server based on Node.js and Express. It has all the basic development server features and includes hot module replacement. Hot module replacement (HMR) is a process where the development server uses a WebSocket connection to replace the code of a given module in the browser when it has changed in the development environment. This does not require a full page refresh to reflect the updates making it seamless to view your changes in the browser during development.

The webpack-dev-middleware npm package is a little more esoteric. It runs as an in-memory store containing the build output as opposed to writing the output to disk. This can decrease the time between when you modify a file and when you can see the changes running in the browser.

Configuration Troubles

Currently in early 2017, webpack is moving from version 1 to version 2. This can lead to incompatibilities in the webpack tool chain. When configuring a webpack build yourself, expect to find out-of-date examples and see errors processing your build.

Not all loaders and plugins support both webpack 1 or webpack 2 and you may find yourself trying different versions of these npm packages with different versions of webpack. Throw in the fact that TypeScript is adding significant new features through all of this and it's a recipe for configuration troubles.

This isn't meant to scare you off, just be prepared. Expect to search through the GitHub issues of some of these frameworks looking for answers or expect to post issues yourself. It's not always clear which combination of settings are correct but the webpack community is by-and-large responsive and committed to this tooling.

This is an example of an issue you may find. While the TypeScript compiler allows comments in the tsconfig.json file as of version 1.8, some of the TypeScript webpack loaders will blow up if there are comments in this file. There is no helpful error message, it just throws an error that a property is not defined on a null reference.

It's likely these issues are resolved over time, but you should understand what you are up against.

Replacing SystemJS

Now that you know about webpack's advantages and potential pitfalls, what are the general steps required to replace SystemJS? This is a summary:

  • Move polyfill scripts from index.html into the webpack bundle
  • Compile and bundle the TypeScript files
  • Move any external component HTML templates and stylesheets inline
  • Add a reference to index.html for the new bundled resource

Move polyfill scripts from index.html into the webpack bundle

Many Angular tutorials instruct you to put polyfill dependencies directly in the index.html (or alternatively named default HTML) file. This approach works and will continue to work with webpack but you should consider moving these dependencies into the webpack configuration.

First, including these dependencies into a bundled request improves performance. Second, having all your script dependencies in one place decreases the time to find a given dependency – thus increasing maintainability. Third, webpack resolves dependencies through node module resolution meaning you add the package name to the configuration and you're done. Using SystemJS, you often must identify the correct script to load from within the npm package.

Compile and bundle the TypeScript files

While you could continue to compile TypeScript with the TypeScript compiler, the idea is to move everything into webpack. There are several loaders that handle TypeScript files and are generally straightforward to configure. A popular TypeScript loader in the Angular community now appears to be awesome-typescript-loader.

Move external component HTML templates and stylesheets inline

This one is Angular-specific. When you have components with external templates and/or stylesheets, Angular loads these files as separate requests. By using a loader like the angular2-template-loader, the build now can combine the contents of the external templates and stylesheets into the component script itself.

Note: Understand that by moving the templates and stylesheets inline, setting the @Component directive's moduleId property is unnecessary. Leaving the moduleId property set in the component can lead to issues. While the available tooling should most likely handle this, you may find you must account for this yourself by removing the settings manually or creating automation.

Add a reference to index.html for the new bundled resource

Finally, once you define the file(s) that webpack outputs, you must reference the new bundled file(s) in your main HTML file. As you may have guessed, there are even webpack plugins to handle this as well based off the webpack configuration (see html-webpack-plugin).

Additional Configuration

While the previous sections listed the necessary pieces of a basic webpack build. You have so many more options available to you to optimize your code. Some build steps to consider are: Ahead-of-time (AOT) compilation, using the wwwroot folder in ASP.NET Core, tree-shaking, running tests, and using the webpack development server with hot module replacement.

Tackling the Beast

So, you still want to get going with webpack? Where do you begin? There are many projects on Github and other sites that provide insight into how this tool works. It appears to still be hit or miss as far as quality. Looking at it from an ASP.NET developer's perspective, the guidance options are fewer. With that said, here are some links to get started:

None of these resources paint the full picture but they provide a good overview. In addition to these general resources, visit the documentation for the individual loaders and plugins to get details regarding compatibility with other packages.

You should consider creating your own webpack build from scratch. There are advantages to doing it this way. If you like knowing how these tools work, this is a rewarding way to learn, albeit a time-intensive one.

Focus on one piece of the build at a time. It's easier to identify which pieces of the build are failing and you can more easily investigate or replace that specific loader or plugin to compare results. Once you stabilize one part, move on to the next until you have the build that you want.

Start with the pieces outlined in the Replacing SystemJS section and continue from there.

More on the Way

You've learned why you should consider moving to webpack and the general approach to creating a webpack-based build. Again, this is just a primer. The plan is to ultimately provide you with a webpack configuration that works well with Angular and with ASP.NET Core. You will see more posts about webpack here soon.

For now, what are your favorite webpack resources? Did you build a template that others may find useful? Please share in the comments.