💡ESLint performance got you down?
Some assorted findings after trying out Biome, Oxlint, and ESLint v9 at Vercel, including 60% better performance(!), a sprinkle of AI, and some exciting results.
Context:
→ We use monorepos. We parallelize and cache our lints with Turborepo. Its not our CI bottleneck, but our IDEs weren't as snappy as they used to be, so some investigation was needed. A clear potential improvement was linting speeds.
→ We use a lot of ESLint rules, for better or worse. I set a goal that I wasn't trying to change the rules we did/didn't have. We could have argued about which rules are valuable until we were too tired to ship any value. I just wanted things to be faster, with no other visible changes to our developers. Non-disruptive as possible.
Experiments:
→ ESLint v9: The latest version of ESLint has a new configuration approach that makes it faster and easier to use. This is good! Configuration discovery is quicker and there's a compatibility package to convert your v8 configuration. After some tweaking, things worked.
↳ Unfortunately, the speed gains I saw were marginal. My editor might have been a little faster - or I could have been imagining it. I'm still not sure.
↳ Speeds in CI were marginally faster, but nothing that I was amazed by. I couldn't see it being a valuable speedup for our developers.
↳ This wasn't the big win I was looking for and my fellow developers don't have great vibes with ESLint after a while of degraded IDE performance. Let's try some new tools.
→ Oxlint:
@boshen_c and co. are doing awesome work over in Oxc-land, so I wanted to give Oxlint a try.
↳ I first needed to find out what ESLint's performance would look like if Oxlint took over all of the work it could. I copied the rules list from the Oxlint webpage into the Vercel AI SDK playground and had it reformat it to an ESLint `rules` configuration object with every rule turned off. I pasted that into our ESLint configurations to get those rules turned off...
↳ This resulted in 40% to 60% faster linting. 🤯 This would be a big deal for our developers! But what about Oxlint itself? Am I just placing those time into a different tool?
↳ After getting a proof of concept of Oxlint setup, it checked 11034 files in 473ms. 🤯 Wow!
↳ I turned on the recommended ruleset to see how my IDE felt and putting in a rule violation resulted in instant diagnostic feedback. We weren't seeing that with the same rule enabled in ESLint.
↳ Two minor shortcomings:
→→ 1) I couldn't find a way to nest configurations well. In a monorepo, there might be different configurations per-package, and I couldn't find what felt like a happy way to do this. I could invoke Oxlint with completely separate configurations per-package, but that felt like I was getting away from how Oxlint wants me to use it, and not very composable. Not a big deal, but keeping it in mind as I continue exploring.
→→ 2) No blessed LSP for our Neovim users. There's a VSCode extension that works great, and we found the LSP source code so we can contribute a plugin back to the Neovim ecosystem if needed. Again, not a big deal, but keeping it in mind.
→ Biome: The
@biomejs team is also doing great work, so worth taking a look at how Biome would work out for us.
↳ Same process as Oxlint. Disable all the rules Biome would cover by pasting the rules list into AI, see what happens.
↳ 35% to 45% faster linting! 🤯 This is, of course, not as big of an improvement. I noticed there were fewer linter rules in Biome, but still a substantial breakthrough.
↳ How fast is the Biome proof of concept? It checks 11493 files in 657ms. 🤯
↳ The IDE performance is also incredible, resulting in instant diagnostic feedback. Similar-feeling to Oxlint, and much different than ESLint.
↳ Shortcomings? Nothing substantial! Biome allows nested configurations, a first-party VSCode plugin, and the Neovim community has built the tooling for integrating Biome. Very cool!
One more thing:
→ Biome also has a built-in formatter, and a few folks requested that I investigate using the formatter as well. The idea of using fewer tools (replacing Prettier) was attractive.
↳ With only a little more configuration in the `biome.json` file, I had a (nearly, see below) 1:1 match with Prettier.
↳ Changing my `biome lint` command to `biome check`, I now had an integrated formatter and linter running on 11493 files in 2 seconds. 🤯
↳ Shortcoming: CSS, YAML, and Markdown aren't supported in the formatter (yet?). This was a material weakness for our repo, so I opted to let Prettier continue running on those files. This isn't ideal, since we wanted to have fewer tools - but it's a step in the right direction.
Decision:
→ After some discussion, the integrated toolchain story won out, despite the faster linting performance from Oxlint. Fewer tools sounds more attractive, even if it isn't going to happen today.
↳ Note: Oxlint has a formatter listed in their future work but I'm working on this today. 🙃
Incremental migration approach:
→ The Prettier to Biome switch is nearly 1:1. We have a small handful of files that I've ignored for now due to minor differences. We also have some `// prettier-ignore`s that will turn into `// biome-ignore`s, but not many. We'll address these in follow-up, to make sure we're keeping our PRs small.
→ The ESLint to Biome switch will involve incremental PRs to turn more and more rules on in short order. ESLint and typescript-eslint will stick around for now, and we look forward to Biome's work on their type-aware linting in particular.
→ I added the new Biome checks to our Turborepo tasks in a few moments. Seamless!
Thoughts and reflections:
→ rust btw
→ It's amazing to think about how the ecosystem progresses and the time savings this could mean at scale, similar to what we see with Turborepo.
→ I'm excited for our developers to feel their IDEs get faster. DX means faster ships!
→ I don't love that I introduced another tool into the repo and the complexity of running two linters and two formatters instead of one for each. However, the tradeoffs are there. We're faster this way, and I trust Biome to keep eating away at the other tools in time.
→ Reading this back to myself, I hope that no one reads this as an indictment of any of these tools, especially ESLint. These tools are all important to the ecosystem in their own ways, even if that way is historical.
→ Hopefully this information helps someone out there improve their DX at their company!