It gets worse when multiple layers of test frameworks are involved. In our current project, I had the problem of writing code (and tests) that is supposed to run on both browsers and Node.js. No problem, right? Just don’t import any Node-specific APIs! We’re already using Rollup as our module bundler, which warns us in case there’s any weird imports.

The reality is of course more complicated. As soon as we write code that interacts with the DOM, we don’t get any warning, because document or window are just global variables. And JSDOM is not always a good replacement because there are some subtle differences (or sometimes even flat-out not-implemented APIs) to actual browser. Additionally, we may be using libraries that import Node modules like assert: they can be polyfilled, but someone has got to do that. Is that the job of the test runner? Of course not.

So, there’s no free lunch. We’ve got to provide a setup to run tests in an actual browser. So far, we had been using Jest, because it provides us with all of what we needed out of the box. But Jest falls short of providing a simple mechanism of running tests in an actual browser;[1] being tailored solely to Node.js.

Enter Karma

Karma is a test runner that was created by the AngularJS team. It provides a versatile plugin infrastructure, which is a double-edged sword. On the one hand, it can be configured to do pretty much anything, including many different reporters On the other hand, it may be hard to figure out how to achieve a certain goal in this large solution space.

So, after some fiddling around, we made some decisions:

  1. Since we’re using Rollup for bundling our code anyway, we’ll be using the karma-rollup-preprocessor to compile the TypeScript code and polyfill APIs.
  2. We’re going to write the tests with Jasmine. That was just a random choice and could have equally well been Mocha.
  3. We want to test on fresh, headless browser instances, so we’re installing puppeteer for Chromium and puppeteer-firefox for, well, Firefox.
  4. We’ll separate our code into three buckets: shared, browser-only, Node-only. Tests files will be specific to one platform (i.e. no test file will be run on both platforms), but they may import from any of the buckets.

All of these decisions could be challenged because they restrict what’s possible in one way or the other. For the purpose of this blog, treat them as assumptions.

Now, before we look at the config, let’s take a brief look at the test files.

Writing tests with Jasmine

Being used to Jest, writing tests with Jasmine is no big suprise:

async function iframePortal(): Promise<Portal> {
    const root = document.createElement("div");
    document.body.appendChild(root);
    // ...
};

describe("RPC through iframe", () => {

    it("Simple communication", async () => {
        const { port1, port2, cleanup } = await iframePortal();

        const promise = receiveSingleMessage(port2);
        port1.postMessage("hello");
        await expectAsync(promise).toBeResolved("hello");

        cleanup();
    });

});

The code sets up an iframe and a MessageChannel and sends one of the resulting ports down an iframe. The precise details don’t matter, but the point is that we’re exercising a whole bunch of browser APIs here. More specifically, browser APIs that are not fully implemented in JSDOM.

The general lifecycle functions (describe, it, beforeAll, …) work exactly the same in Jest and Jasmine. The matchers however look slightly different. In the future, we might standardize on a third-party matcher library (like Chai) to write our assertions in a uniform style.

Similarly, the Node.js tests can be written in exactly the same way and may also import Node APIs.

Executing tests in the browser

The plan is: run all tests matching the file name src/tests/browser/**/*.test.ts in the browser. For that, we need a few packages (please bear with me):

Now, let’s jump into the config. Make sure you’re sitting, because it’s a mouthful.

const sucrase = require("@rollup/plugin-sucrase");
const resolve = require("@rollup/plugin-node-resolve");
const commonjs = require("@rollup/plugin-commonjs");
const builtins = require("rollup-plugin-node-builtins");
const globals = require("rollup-plugin-node-globals");

process.env.CHROME_BIN = require("puppeteer").executablePath();
process.env.FIREFOX_BIN = require("puppeteer-firefox").executablePath();

module.exports = function (config) {
    config.set({
        basePath: "",
        frameworks: ["jasmine"],

        files: [
            "src/tests/browser/**/*.test.ts",
        ],

        preprocessors: {
            "**/*.ts": ["rollup"]
        },

        rollupPreprocessor: {
            plugins: [
                globals(),
                builtins(),
                resolve(),
                commonjs(),
                sucrase({
                    exclude: ["node_modules/**"],
                    transforms: ["typescript"]
                })
            ],
            output: {
                format: "iife",
                name: "postoffice",
                sourcemap: "inline"
            }
        },

        browsers: ["ChromeHeadless", "FirefoxHeadless"],

        reporters: ["spec"],
        port: 9876,
        colors: true,
        logLevel: config.LOG_INFO,
        autoWatch: false,
        singleRun: true,
        concurrency: Infinity,

        client: {
            captureConsole: true
        }
    })
};

But what does it do?

  1. Files ending with .ts (i.e., all test files) are processed through Rollup.
  2. Rollup bundles all dependencies into a single file, including other packages from node_modules.
  3. Sucrase is used for fast compilation from TypeScript to JavaScript without type checking.
  4. The result is emitted as an IIFE together with a sourcemap so that we still know where errors came from.

The other parts of the config are fairly standard.

Executing tests on Node.js

For running the tests on Node.js, we can’t use Karma. Sure, there’s a launcher for JSDOM, but we want to run tests without access to browser APIs. Hand-rolling is hard too, because Karma doesn’t serve the tests directly; rather, they’re wrapped in HTML code. So, we decided to use the Jasmine launcher directly since we’ve already written our tests with Jasmine.

Again, we’ll need a few additional dependencies:

Note the absence of any Rollup configuration here. The reason is simple: we’re running on Node, with full access to node_modules, so there’s no need to bundle the test files. However, we do still need to compile TypeScript to JavaScript. In Jest, the recommended way to do that is with Babel. For Jasmine, we were unable to find out the best practices, so we settled on ts-node which acts as a wrapper for the Node.js binary and compiles TypeScript on the fly.

Jasmine itself is opinionated about where test files and its configuration file should live. Luckily, this can easily be configured away. Here’s the jasmine.json file:

{
  "spec_dir": "./src/tests/node",
  "spec_files": [
    "**/*.test.ts"
  ],
  "helpers": [
    "jasmine/*.ts"
  ]
}

The helper file just sets up the jasmine-spec-reporter:

import {SpecReporter} from "jasmine-spec-reporter";

jasmine.getEnv().clearReporters();

jasmine.getEnv().addReporter(new SpecReporter({
    spec: {
        displayPending: true
    }
}));

Finally, we wire this up as a script in package.json:

"scripts": {
    "test:node": "ts-node ./node_modules/jasmine/bin/jasmine --config=jasmine.json"
}

That’s it! Now, Jasmine can test Node code (through ts-node) and browser code (through Karma and Rollup).

Other complications

In our case, there were a few other complications.

The browser tests needed to set up an iframe that runs some JavaScript code, so we had to provide these files (HTML and JS) too, but without it being run as a test suite. Luckily, Karma understands that concept:

files: [
    "src/tests/browser/**/*.test.ts",
    { pattern: "src/tests/data/**", included: false }
]

We can declare some file patterns to be processed, but not included in the test run. They can be used in the tests as follows:

const iframe = document.createElement("iframe");
iframe.setAttribute("src", "/base/src/tests/data/iframe.html");

/base is the route prefix used by Jasmine’s default middleware. The frame itself can reference a JavaScript file that’s generated from TypeScript code:

<!DOCTYPE HTML>
<html>
    <body>
        <script src="./iframe-script.js"></script>
    </body>
</html>

Finally, we also needed to access some other web resources from within the browser tests. Karma allows extending its web server with custom middleware, which makes this quite easy. The Karma config can be extended as follows:

// import the middleware
const {KarmaRPCPlugin} = require("./karma/dist/index");

// in the config
plugins: [
    { "middleware:rpc": ["factory", KarmaRPCPlugin] },
    "karma-*"
],

beforeMiddleware: ["rpc"]

Now, our browser code can access the resource /rpc that is routed to the KarmaRPCPlugin.

Conclusion

Even though the configuration overhead is significant, I’m fairly happy with the outcome. We can test the exact same TypeScript code uniformly across multiple browsers and Node.js. I’m not entirely confident though that this machinery can be reasonably extracted to a general-purpose cross-platform test runner. But it would make sense to standardize this setup for some organisations to get consistent tests across multiple packages.

  1. I‘m aware that there’s integration between Jest and Puppeteer; however, this merely provides a Jest environment in which Puppeteer is available. There's no direct support for running JS code in the puppeteered browser, short of hand–rolling a web server.  ↩

TAGS