Since we released Intern in 2013, the state of JavaScript testing has changed substantially. The JavaScript world was a very different place. Node.js was at version 0.8, and core modules like http were still being heavily developed! Popular tools like webpack, React, and TypeScript were still in the early stages of development. Jasmine was probably the most popular testing framework for JavaScript, although there were were several other tools in common use such as Mocha and QUnit. WebDriver testing with JavaScript was still a fairly manual affair unless you used a cloud testing service like Sauce Labs or BrowserStack.

Four years of Intern

When we first introduced Intern, our assessment of the testing landscape was “It’s a mess”.

While several testing tools were available at the time that generally covered the needs of testers, developers had to stitch together a complete testing system from a bunch of different places. Intern was created to alleviate this problem and to provide a solid glue architecture for testing applications using JavaScript and various asynchronous patterns such as Promises. It brought together many of the best features from a number of different tools (and sometimes entire tools) into an integrated package:

  • Support for modular AMD code in Node.js and web browsers
  • Integrated code coverage analysis and reporting via Istanbul
  • Ability to run both unit tests and functional (WebDriver) tests
  • Rich assertions via the Chai library

Having all of these features in one place streamlined the approach to writing a full set of application tests, from unit to functional to end-to-end. Intern has certainly helped us and our users test their applications far more efficiently than was previously possible. As the world of JavaScript evolved, though, we noticed that writing quality tests was getting more difficult.

The changing landscape

The JavaScript ecosystem, and even the language itself, have undergone significant changes in just the last few years, which has created new requirements and challenges for JavaScript testing.

JavaScript the language

ECMAScript 5 had become a standard in 2009, and by 2013 it was mostly implemented in browsers. Today the language is moving much more quickly, with new JavaScript specifications published yearly starting in 2015. The most widely used browsers, Chrome and Firefox, are on rapid release schedules with many users being automatically updated, so new features are making their way to users more quickly. Transpilers such as Babel allow engineers to leverage the latest language features until browser support is complete. These are certainly exciting and promising times for web developers and engineers!

With these advances, particularly the increased use of transpires, source maps have become much more important. A few years ago source maps were mostly needed to map errors in minified code back to source, which often was not applicable in testing scenarios. However, with source code now often using ES6+ syntax that isn’t natively supported by all browsers, not to mention language extensions such as TypeScript, testing tools need to support source maps to be able to map assertion failures in running tests back to the source being tested, and to provide code coverage analysis against the original rather than transpiled source.

Static typing

Static typing has also become more accepted in the JavaScript community. Microsoft debuted its TypeScript superset of JavaScript in 2012, and by 2015 it had reached sufficient maturity that Dojo, Angular, Aurelia and others were all looking to leverage the language for part or all of future versions of their frameworks. The React team was also interested in static type checking and introduced Flow, which implements types using comments rather than new language syntax. Adding types to a project with Flow is typically easier than with TypeScript, but Flow isn’t as capable as TypeScript in some respects (it’s not a transpiler), and its IDE support is more limited.

While static typing can complement testing in some sense by catching syntax and API usage errors earlier, it also creates new testing challenges. For example, typings created for a pure JavaScript library should ideally be validated to verify that they properly describe the library.

Module loading

When Intern was first created, there was no language-level standard for JavaScript modules, and AMD was the best way to efficiently load modular code in the browser. Intern supports CommonJS to some extent, but test suites were most commonly written as AMD modules to allow these tests to be run in both Node.js and web browser environments. While AMD is still commonly used today, it is less necessary for loading modules in the browser. Alternative loaders, such as SystemJS, can load CommonJS and ES modules (as well as AMD), both in Node.js and the browser. Bundlers such as webpack have also become very popular for enabling modular code to run in browsers (among their many other uses). ES6 added a new syntax for defining modules, though the standard for loading ES modules is still being actively refined and is just starting to land in browsers in 2017.

Testing today

While the landscape has been changing for the better, it is clear that several of these changes present new challenges in the present and near future. JavaScript testing tools have not been sitting still for the last few years, and help provide solutions for this changing landscape.

Testing tools

It is clear that several of these changes present new challenges in the present and near future.

In 2014 Jasmine gained Node.js support, and significantly improved its support for asynchronous code, and it remains one of the most commonly used unit testing frameworks. Mocha has become increasingly popular over the last few years as well, adding numerous features such as support for Promises. Newer frameworks have come along, too, like Jest and AVA. Jest is a unit testing framework developed and used by Facebook with a focus on easy setup and good support for React projects. Like Jasmine, Jest includes a number of support tools like mocking and assertions, and Jest also includes code coverage support. AVA is an opinionated, lightweight testing framework that focuses on speed. It concurrently runs test files, each in a separate process, improving both execution isolation and throughput.

Overall there is no shortage of useful unit testing frameworks in JavaScript!

Functional testing with WebDriver has improved significantly in the last few years. Version 3 of Selenium was released in 2016, and the WebDriver protocol reached W3C candidate recommendation status in 2017. Nightwatch and Webdriver.io are two newer frameworks that leverage WebDriver for functional testing. Both of these tools include their own testing interfaces and test runners, and both can also use the reporters and interfaces of some popular unit testing frameworks like Mocha. Intern’s Leadfoot also continues to be a viable option for functional testing, with its unique feature testing approach to help prevent WebDriver failures due to inconsistent implementations of WebDriver.

Tools that support the testing process have also continued to advance. Istanbul, the code coverage instrumentation and reporting library, has undergone a complete rewrite. It leverages the nyc command-line tool and also works with non-transpiled ES2015 code. Testing tools can once again generate stack traces for transpiled code that point back to the original sources, with the features of remap-istanbul built into the new version of Istanbul. Headless testing of browser APIs using either more recent versions of PhantomJS or jsdom have improved but still do not replace the efficacy of testing against the real browsers in which your applications will be run.

One aspect of JavaScript testing that has not changed much in the last few years is the dearth of tools allowing both Node.js and browser-based unit tests, and functional tests, to be created using a unified workflow. The burden still falls on developers to stitch together a complete test system for their applications. This approach to creating a consistent test stack remains our primary motivation behind Intern.

Intern

Intern
Intern has evolved significantly over the last several years, continually adding new features to make writing (and running) tests simpler, from small improvements like glob support for listing suites, to more significant features like the ability to efficiently manage a local Selenium server, as well as automated accessibility, performance, and visual regression testing. These improvements and the hundreds of others were very welcome for improving how we test our applications, but did not address some of the major changes in testing that are possible and needed with the introduction of ES2015+.

The upcoming Intern 4 addresses these changes and challenges with a number of updates that improve its support of today’s JavaScript:

  • Intern has been completely rewritten in TypeScript and provides a thoroughly typed public API. The typings also make contributing to Intern easier by helping to ensure that changes properly use internal APIs!
  • Intern is now module loader agnostic, allowing tests, and application code, to use any module format (or no module format), including AMD and ESM.
  • Source map support has been significantly improved, allowing Intern to track locations through instrumented and transpiled code.
  • Code coverage support has likewise been upgraded to handle transpiled code.
  • The configuration process is simpler and more consistent, and less configuration is required to get up and running.

Intern continues to be one of the only testing frameworks with out-of-the-box support for both unit testing (in Node.js and browsers) and functional testing, maintaining its goal of making writing quality tests (all kinds of tests) easier! And Intern 4 will now make this benefit easier to leverage with very modern JavaScript and TypeScript patterns and best practices.

Testing tomorrow

The pace of JavaScript development shows no signs of slowing down. In fact, with the recent move to yearly language standard updates, it is still speeding up! As the language and its ecosystem changes, so do the capabilities required of testing tools. We strive to make sure Intern provides those capabilities, and make it a great fit for many projects. No solution is one-size-fits-all, though, and luckily there are several great open source testing tools to help you with your next project. These tools provide almost everything you need, other than writing your tests for you!