JavaScript code quality with free tools

As I mentioned in the previous article, we’re working on a simple caching GraphQL client with all of the features you want, including query diffing, pagination, optimistic UI, and more. You can follow our adventures on the Building Apollo publication here on Medium.

The mental model is pretty simple: it just takes some JSON data returned from a GraphQL server and puts it in a normalized store. But since the implementation necessarily involves manipulating parts of GraphQL syntax trees and passing around odd data structures, every new feature or refactor risks breaking parts of the codebase. Working with the awesome Apollo contributors James Baxley III and Maxime Quandalle, I decided to take some time to sharpen our tools so that we could write code faster and worry less.

Here’s what we ended up doing to set up static typing, linting, CI, and test coverage reporting for our in-development npm package:

Static typing

With TypeScript

One of the surest ways to get the computer to check your code for you is to instrument it with static types. If you add the right type annotations to your code, you can make sure that all of your internal functions are passing the right types of data to each other, without having to write endless unit tests for all kinds of malicious input. But that’s not all — you can also get the type system to help you autocomplete function calls and property accesses, remind you which parameters to pass, and generally make your code more self-documenting for future developers.

Unfortunately, JavaScript doesn’t come with static typing out of the box, so we had two options to get this working:

1. Flow
2. TypeScript

We had a few discussions about the tradeoffs. We were pretty confident that our basic type checking needs would be addressed by both systems, so the decision came down to several main properties:

1. Editor tooling for type checking, refactoring, and auto-completion: Flow works with an Atom plugin via Nuclide, and TypeScript has its own open source IDE in Visual Studio code, as well as editor plugins for Atom and other editors. In this category, TypeScript was a clear winner, with a completely seamless setup with an IDE that pretty much worked out of the box.
2. Ecosystem of typed packages: Flow has the advantage that the reference GraphQL JavaScript implementation is written with it, as well as Relay, Facebook’s GraphQL client framework. TypeScript is used in many important libraries as well, for example Angular 2 and Immutable.js. The clincher here was the huge variety of type definitions available online, and even a command line tool for managing them called Typings. We were able to quickly install a TypeScript definition for GraphQL-JS, which has been working really well so far.
3. Coupling of compilation and type checking: TypeScript positions itself as a new language, while Flow is a type checker for JavaScript. Essentially, this means that TypeScript has one tool that both compiles and checks the code; Flow lets you compile the code using Babel, and has a completely separate checker that runs in a different step. This means that Flow lets you use whatever Babel transforms you want to, which is a pretty big benefit. TypeScript doesn’t support as many language features, and you have to wait for a new version of TypeScript to come out to get them.

At the end of the day, points (1) and (2) won over, and we decided to accept using only the JavaScript features provided by TypeScript to get access to the integrated tooling and huge library of type definitions available.

It has been working really well so far, and the integration between Visual Studio Code and the type system has been a real treat. We also set up TSLint to do some basic linting to make sure our code style remained consistent.

Testing and continuous integration

With Travis CI and AppVeyor

Running tests in TypeScript is exactly the same as with normal Node and NPM — you set up a test script in your package.json file, and use a test runner to execute some JavaScript code. We simply had to run the TypeScript compilation step, and from there it was smooth sailing with source maps giving us stack traces with the real line numbers. Here’s the relevant excerpt from our package.json:

// Our NPM test scripts
"scripts": {
  "pretest": "npm run compile",
  "test": "mocha --reporter spec --full-trace lib/test/tests.js",
  "posttest": "npm run lint",
  "compile": "tsc",
},

Now we had to make sure the tests actually ran all the time, and we could see the status on pull requests and different branches. We ended up setting up two testing CI services:

1. Travis CI has become one of the go-to options for open source projects, because of their generous free offering that lets you run parallel tests on different Node versions and great integration with GitHub.
2. AppVeyor was added when we realized that certain parts of our build setup didn’t work cross-platform. Over time, we realized that the free tier on AppVeyor was running so slowly that we spent a lot of time waiting for PR builds to finish when Travis was long done, so we switched it to only run for the master branch.

This is pretty standard stuff for Node projects, but its value can’t be overstated. Check out our repository for all of the config files.

Test coverage

With istanbul and Coveralls

Now we were in a pretty good spot — we had type checking and linting catching most of the silly typos, continuous integration running the tests to make sure we didn’t break anything, but something was missing — how do we actually know that our tests are exercising all of the code? If someone wanted to add new tests, how could they learn what parts of the code could be better tested?

Clearly, there is no foolproof way to check that your tests are testing everything you might want to know about your code, but you can get better than nothing by using a test coverage tool. I decided to go with a tool called istanbul, mostly because I had heard good things about it before.

It took a little effort and iteration to set up the right scripts to get reporting working properly:

// Our NPM test coverage scripts
"scripts": {
  "coverage": "istanbul cover ./node_modules/mocha/bin/_mocha -- --reporter dot --full-trace lib/test/tests.js",
  "postcoverage": "remap-istanbul --input coverage/coverage.json --type lcovonly --output coverage/lcov.info"
},

The biggest issue I ran into was with source maps. Since we were writing our code in TypeScript, then compiling to JavaScript, then istanbul was transforming it again to instrument the code for coverage analysis, it was actually messing up the stack traces when our tests threw errors. The simplest way to fix this was to run the tests twice: Once to actually test the code, without istanbul, so that we can get good traces, and then a second time just to get the coverage report. Since we don’t need to run the coverage report very often, that has been a fine solution so far.

But even though we set up a test coverage tool, that still didn’t help easily answer the question — where should we write more tests? It would be ideal to just have a dashboard where we can browse the code and see the coverage… and that’s exactly what Coveralls does!

Once we set up our scripts to apply the source maps to the coverage report, and send that data to Coveralls, we were able to get an easy to browse page to show us exactly which statements in the original TypeScript code were executed during the test suite. Now, we can track when we fall behind on test coverage, which edge cases in our code the tests never hit, etc.

For example, now we know that we don’t have any test cases where there is an array in the store, and a query diff operation returns any missing selection sets:

Better add a new test there!

Summing it up

Now, I feel pretty good about making changes to the code, after setting up:

1. Static type checking,
2. Linting,
3. Continuous integration testing on multiple platforms, and
4. Code coverage analysis.

When I do a refactor, or add some arguments to a function, or write some new features entirely, I can be a lot more confident that it works with the rest of the code, is well-tested, and is less likely to break in the future. I also hope it gives the future users of the Apollo Client, when it is released, more confidence that they are using a package they can really rely on.

Written by

Sashko Stubailo

Read more by Sashko Stubailo