Little bit of the TypeScript toolchain
Our monorepo has two major directories. One for our backend services written in Go and one for our frontend clients written in TypeScript. For the past month, I have been putting a lot of hours in to trying to raise the quality of the CI/CD process of the TypeScript to that of Go.
If you have worked with Go, it shouldn’t be too stunning that the our CI/CD process for our backend services is solid. Go comes with a pre-built and highly opinionated toolchain which covers many of the responsibilities a CI/CD system attempts to enforce: code linting, running tests, and consistent builds (dependencies, executables).
go fmt handles linting.
go test is a sufficient test runner. And while Go’s new module system is still ironing out the kinks as of Go 1.18 (like this annoying
go install limitation to install executables which use replace statements) it is able to get the job done and is fairly straight forward. It also has great cross platform compilation builtin. All of these tools also work great in a monorepo. The stability allows other tools (like IDE’s) to easily build on top and provide value for developers. It’s just really great.
Our TypeScript tooling isn’t at Go’s level yet, but its a little better than when we first started.
We roll with
learna) and its hard to say which one is the best. But
yarn has worked for us and is probably the least painful aspect of our tooling.
yarn keeps our 3rd party dependency management maintainable, locking things in for builds and guaranteeing what you would expect.
yarn’s workspace feature makes supporting monorepo development possible. And while some things are left very open ended (like script naming convention), it gets the job done.
There are many test runners out there. We are using
jest which appears to be the most popular, but who is to say. This is where we gotta sink in to the disjoint issues in the toolchain. So
jest doesn’t understand TypeScript, but it does have support for compiling TypeScript and then running tests. This support is
jest specific and requires its own configuration. Compare this to Go’s toolchain, where each step (e.g.
build) can leverage to work done in other steps like compilation. It doesn’t matter if we just linted our TypeScript codebase and did a lot of compilation work,
jest is gonna do it all over again cause there is no way (currently) to share that info with it. This obviously leads to slow performance and a maintenance burden.
jest has a “multi-project” feature which is sold as monorepo friendly and sounds cool on paper, like offering test coverage aggregation across projects. However, its really a much more abstract feature that allows for different configurations of any kind (e.g. different code, different runners, different settings) to be run concurrently. Its actually really difficult to share configuration between these “projects”. So our monorepo contains ~20 packages and would require the same number of hand configured jest configurations. Not maintainable or robust.
Maybe there is another runner out there which would better support our monorepo and TypeScript needs.
go fmt tool to just cover stuff like “how many spaces here”, which is still valuable cause developers never talk about it now, but very small scope compared to its TypeScript equivalent.
ESLint is highly configurable and projects can inject all sorts of logic into it. From what I have seen, it is pretty much an essential tool for developer productivity.
Ideally, a lint process can be constantly giving a developer feedback. If you type some Go code in an IDE it will almost instantly give you feedback if you misstep.
ESLint has similar integrations with IDEs, however, it is not nearly as performant as the Go equivalent. This became obvious quickly even in our modestly sized monorepo.
ESLint processes were sucking up over eight gigs of memory. The information it reports is incredibly valuable, but we had to do some tuning to make it more actionable. We ended up adding an
.eslintrc configuration per package. This keeps the memory usage down. And luckily, unlike
jest, the configuration is almost 100% shareable so its easy to maintain across the repository.
One other pain point with
ESLint is TypeScript dependencies.
ESLint is able to use type information to provide more valuable feedback, it is not integrated into the TypeScript build process though. What this means is that in order to get the feedback, the linter is dependent on an out of band tool (like
tsc) to compile internal dependencies. This is another issue of the disjoint toolchain which I took for granted not dealing with in Go.
As I mentioned up top, Go has great cross platform compilation support. Its easy to target different platforms and trust that things will “just work”. We use it all the time internally building tools which run on linux machines as well as macos boxes.
We are currently using
webpack to get the job done, but are looking in to switching to the more performant
esbuild. No matter which we end up with, as with the testing and linting tools, these bundlers run in isolation from the other tools. Often re-doing work in their own special way. There doesn’t seem to be much work into changing this anytime soon. But other than
webpack being slow as shit, it hasn’t caused much issues. As of today, the most developer productivity pain comes from the linting and testing tools. Hopefully it will change some day.