Migrating millions of lines of code to TypeScript

2022-05-21
Migrating millions of lines of code to TypeScript

On Sunday March 6, we migrated Stripe’s largest JavaScript codebase (powering the Stripe Dashboard) from Flow to TypeScript. In a single pull request, we converted more than 3.7 million lines of code. The next day, hundreds of engineers came in to start writing TypeScript for their projects.

Seriously unreal. I remember a short time ago laughing at the idea of typescript ever landing at Stripe, and then I woke up Christmas Monday morning and it was here.

Mike Fix, Engineer, Stripe

TypeScript is the de facto standard for JavaScript type checking, and our engineers have been overjoyed by this migration. We’re sharing our TypeScript conversion tool on GitHub to help others perform similar migrations.

A brief history of JavaScript type checking at Stripe

Stripe has built large-scale frontend applications since 2012, including stripe.com, Stripe JS, and the Stripe Dashboard. As our company grew, we increased the quality and reliability of our products by type checking our JS code. In 2016, we were an early adopter of Flow, an optional type system for JavaScript developed at Meta (then Facebook). Since then, Flow has provided type safety for the majority of our frontend applications.

However, engineers had trouble working with Flow. The type checker’s memory usage would lock up laptops, and the in-editor integration was frequently slow and unreliable. Meanwhile TypeScript, an alternative type system developed at Microsoft, exploded in popularity thanks to its tooling and robust community. TypeScript availability became a top request among engineers at Stripe.

Stripe’s developer productivity team aims to provide our engineers with the most productive development environment of their careers, and delight in our tools is crucial for that. We work hard to identify the most pressing issues affecting developers; for example, we’ve built integrations into all of our development tools for reporting friction, which is quickly routed to the responsible teams and prioritized. TypeScript support was one such pressing issue and teams supporting frontend engineers began to plan out supporting TypeScript across the company.

Choosing the right migration strategy

Our largest frontend codebase powers the Stripe Dashboard and other user-facing products. The Dashboard codebase has tight coupling between disparate components and no cleanly factored dependency graph. An incremental migration to TypeScript would force developers to work in both languages to accomplish common tasks. We would also need an interoperability layer to sync type definitions between both languages and keep them consistent throughout the development process.

In late 2020, we formed a new horizontal JavaScript Infrastructure team: a group of engineers solely focused on elevating the experience of writing JS at Stripe. One of the team’s first challenges was to replace Flow with TypeScript without a long and uncertain migration.

We began by speaking to companies who had run similar migrations and read articles from Airtable and Zapier describing their experiences. These companies developed automated scripts to convert one language to another, ran them over their entire codebases, and merged the output as a single commit. Airtable had published their conversion script to GitHub as a source-to-source conversion tool, or “codemod,” that would parse Flow code and generate TypeScript.

Migrating in this way would greatly reduce the cognitive overhead for engineers, who would not need to handle both type systems for the same product behavior. We could have a clean break between Flow and TypeScript.

Planning, preparation, and iteration

We were really impressed by the quality of Airtable’s conversion code and decided to use that as the basis for our migration efforts. Many thanks to the team at Airtable for building this out and sharing their work—the open source community benefits a ton from examples like this.

We began by copying Airtable’s codemod to Stripe’s monorepo to run against our internal code. Our JavaScript projects make heavy use of Sail, a shared design system of strictly typed React components, so that was our initial area of focus. We generated TypeScript definitions for Sail, rather than converting the code to TypeScript, as it would continue supporting applications written in Flow. To safely support both type systems, we wrote tests to verify the TypeScript definitions against any changes to the underlying Flow code. This approach would be too cumbersome for a large codebase, but thankfully the Sail component interface is explicit and quite rigid. 

The core of the codemod was solid but not comprehensive: for many files, it would crash or generate imperfect output. Over several months we iterated to handle more syntactic and semantic edge cases.

For one simple example, JavaScript arrow functions can return a single expression without a return statement, such as the following:

JavaScript object literals use braces to wrap property definitions. Because braces are also used to delineate blocks of statements, returning an object literal from an arrow function requires an additional set of parentheses to disambiguate:

We noticed that the codemod was incorrectly stripping the extra parentheses from these arrow functions, but only in the case of a generic function (a function that takes a type argument), which is syntax not available in standard JavaScript:

We were able to fix this issue and add tests to prevent further regressions: There were dozens of similar syntactic fixes we made to handle the breadth of our codebases.

Once Sail was usable from TypeScript, we worked on a couple of internal applications containing hundreds of JS modules. We also added a second pass to the codemod to suppress errors in the generated code, using TypeScript’s @ts-expect-error comment to tag these errors. Rather than resolving every error ahead of time, we focused on eliminating Flow as soon as possible, tracking TypeScript error suppressions to address after the conversion.

An initial pass on the Dashboard codebase created over 97,000 error suppressions. With our iterative approach to updating the codemod, we were able to get that number down to 37,000, or about one per thousand lines of code. For comparison, the Flow code had about 5,000 error suppressions. Both Flow and TypeScript support measuring type coverage, and we were pleasantly surprised that TypeScript reported higher coverage than Flow even with these suppressions. We attribute that to an increase in the number and quality of third-party type definitions available in TypeScript, the lack of which was a large contributor to poor type coverage in Flow.

As we moved onto the Dashboard with its tens of thousands of modules, our approach created significant memory pressure on the TypeScript compiler. Our primary tool to address this was TypeScript project references: Although the Dashboard is not structured as distinct modules, we could infer a module structure and create project references based on that. This approach gave us the headroom to run TypeScript over the codebase without refactoring large chunks of application code.

Going live

Hundreds of engineers contribute to the Dashboard each week. Such a sweeping change would be exceptionally challenging to merge on a normal working day. Our team decided to commit to a date—March 6, a Sunday—where we would lock the Stripe monorepo and land our branch.

In the week before merging, we focused on passing a build through our CI system and deploying it to our QA environment. Although TypeScript could successfully check the project, other tools that process our source code (ESLint, Jest, Webpack, Metro) would also need updates.

One particular pain point was Jest snapshot testing: Jest generates snapshot files with a hardcoded reference to the test file that generated them. Since the codemod would generate either or extensions for TypeScript files, the snapshot files would have invalid references back to their test sources. We simplified this by switching the generation to only use . This meant we could rewrite the snapshots in bulk and keep 100% of those tests passing.

In some cases we recognized that fixing the code for TypeScript compatibility would add weeks to our schedule. One set of cases was our custom ESLint rules: We had a rule to reorder imports to enforce consistency between files, but the rule was written against Babel’s Flow parser, which generated a subtly different abstract syntax tree from the TypeScript parser. In cases like this, we opted to disable some checks and do the work to restore them after the conversion.

With a passing build in hand, we reached out to product teams with user-facing functionality in the Dashboard. Although the Dashboard has extensive unit and functional testing, it has limited end-to-end test coverage. This made manual tests by product stakeholders crucial. Those tests highlighted some minor bugs, which we resolved during the final week: In one case, we were failing to load any translations for non-English Dashboard users, due to a hardcoded extension in the translation loading code.

This process gave us high confidence, but there is always uncertainty with a change this large: Although we had a firm grasp on our developer tooling and build processes, we were mutating every file in the codebase. Subtle errors in our conversion scripts (for example, removing an empty field from an object shared between multiple components) could cause user-facing errors, without being covered by any of our existing automated tests. These failures could manifest in a number of ways, from downstream dev tooling issues to builds that fail. We leaned on our deploy automation and ambient monitoring to make us aware of any unexpected problems, and created a Slack channel to coordinate the rollout so user-facing teams could quickly escalate any reports they would receive.

On Saturday, March 5 the team generated a new migration branch and ran our automated scripts. We then deployed that branch to QA and repeated our validation process, including the manual tests suggested by product teams. We found no new issues. We were ready for the day of the merge.

Early on the morning of Sunday, March 6, we locked the Stripe monorepo, took one more QA pass over our migration branch, and submitted the change. It merged cleanly and our automated tests passed. We kicked off the deployment to ship TypeScript into production.

Thanks to the care and rigor of the previous year of work, we had no unpleasant surprises as we shifted traffic to the new code. We unlocked the repository and let developers know that the Dashboard was now in TypeScript.

When I was interviewing, I heard the migration from Flow to TypeScript was underway.

I was admittedly skeptical, seeing prior teams struggle with the complexity and effort of even small codebases.

The fact that I was back to normal in a few minutes [on] Monday was humbling.

Eric Clemmons, Software Engineer, Stripe

The immediate response was overwhelming. Engineers were impressed by the completeness of the migration: one described it as the single biggest developer productivity boost in their time at Stripe. We were happy to have the year of work pay off with such a clear and dramatic improvement to Stripe’s codebase.

TypeScript… two months later

The conversion was not perfect. Over the subsequent weeks our JS Infra team addressed issues as they arose. One example we didn’t anticipate was engineers reporting inconsistency between CI and local TypeScript runs. In TypeScript we are able to use many third-party type definitions installed from npm, and if those are updated, engineers will need to install the new versions. This was different from our Flow configuration, where dependency updates rarely changed types, so we had to educate engineers to try running as a debugging step.

There is still more work to be done: We know performance could improve further with more granular project references, and better caching could speed up our CI runs. However, the benefits have far outweighed the bumps along the road. Engineers enjoy features such as automatic dependency imports and code completion, as well as the TypeScript community’s extensive corpus of third-party type definitions and integrations. When new engineers join Stripe to write frontend code, from day one they can be successful in a language with which they’re more likely to be comfortable and familiar.

With the work on the Dashboard complete, the JS Infra team has continued to increase TypeScript’s adoption across the company. We’ve used the same tools to convert many other codebases, including all of our Payments UIs, such as Stripe Checkout. Stripe frontend engineers will soon write TypeScript for whichever project they develop.

When we first shared the story of our migration publicly, the response was equally enthusiastic. Developers from across the industry reached out to learn more and apply the same improvements to their own codebases. To support these developers, we’re sharing our TypeScript conversion code on GitHub for teams to adapt to their own projects.

Aside from the particulars of JavaScript or Flow or TypeScript, our big lesson from this migration is that dramatic improvements to large codebases are possible with diligence, commitment, and optimism. We will apply that mindset to other opportunities to make our engineers more effective and hope others do the same.