migrating millions of lines of code to
typescript on Sunday March 6th we
migrated strip's largest JavaScript
codebase powering the stripe database
from flow to typescript in a single poll
request we converted more than 3.7
million lines of code the next day
hundreds of Engineers came in to start
writing typescript for their projects
that's a big ass PR seriously unreal I
remember a short time ago laughing at
the idea of typescript ever landing at
stripe and then I woke up Monday morning
and it was here thanks Mike fix
typescript is the deao standard for
JavaScript typechecking and our
Engineers have been Overjoyed by this
migration we are sharing our typescript
conversion tool on GitHub to help others
perform similar migrations okay okay so
they are obviously going from flow to
typescript by the way this should be
just a good little mental note for
everybody the next time you're hyped up
about the greatest and latest JavaScript
thing just remember that for every
person that gets hyped up about some
latest and greatest typescript
JavaScript thing there's a company 5
years down the line trying to figure out
how the hell do we get off of this old
ass janky piece of system that
somebody you know that was great at the
time but now sucks just remember that
just remember that most types or most
react today are still classes Legacy
systems are real people functional
components came out like 8 years ago and
people are still just it's just classes
classes everywhere I think a lot of
people forget just how long software can
be around when you write software you
write it as if it's going to be around
for a year and 90% of the time you're
right the other 10% it's around for 20
years and you have some that I mean
Cobalt still exists for a reason today
people if you if you just don't believe
that just remember go look at Cobalt and
then you'll be then you'll remind
yourself oh yeah that's why stripe has
built a large scale frontend application
since 2012 including stripe.com
stripe.js and stripe dashboard as our
company grew we increased the quality
and reliability of our products by
typechecking our JS code in 2016 it
should I mean should put you should put
the word linting actually okay let me
just put my Fedor in 2016 we were an
early adopter of Flo an optional type
system for JavaScript developed at meta
then Facebook since then Flo has
provided type safety for the majority of
our frontend applications I have never
actually really seen much about flow is
this flow well I mean so far it it looks
the same I wonder what that plus sign
means example of generating flow type
from an API resource and Associated
endpoints okay however Engineers had
trouble working with flow the type
Checker's memory usage would lock up
laptops there's something so beautiful
about this I mean remember typescript is
is huge all right when you run
typescript like if you run just right
now if you go and run two separate files
on just with typescript you'll probably
use two three gigabytes of memory
unironically here I think I have this
thing around here just test yeah I can't
remember how to how to actually run this
bad boy cat package was I smart enough
to leave myself any bread crumb no I
wasn't let's go like this npx just oh
yeah oh yeah oh yeah I left it around oh
yeah we left it around boys all right so
I'm going to take this thing I'm going
to grab I'm going to grab this little
thing right here don't worry it runs for
45 seconds it has a 45 second wait
personal V VM RSS VM RSS paste that in
there you go look at that here's the
best part is this is like a generated
small little project right here I R I'm
running two tests look at all those
empty node instances for no reason this
is just how many cores my computer has
it's 1.1 GB on on nothing it's two tests
plus a whole bunch of just empty things
running this is what just runs this is
just run just which runs a shell which
runs node which runs the amount of
processes you have as nodes the in
editor integration was frequently slow
and unreliable meanwhile typescript and
alternative type system developed at
Microsoft exploded in popularity thanks
to its tooling and robust Community
typescript availability became a top
request among engineers at strip I mean
typescript the the LSP for typescript is
pretty good it's fast it mostly works
there's some gripes one could have about
it like one thing that's really like if
you've never worked in another language
you may not realize some things that are
really awesome and so one thing that you
may not know if I'm in here and I'm
working on a project right here right
and I delete this and I run trouble I
will get every single file in which I
just
broke okay here's every sing file that I
just broke with that thing and some
breaks that I just have if you're in
typescript land what's going to end up
happening is that only the files you've
like told typescript about are they
going to do and if you have a smarter
system that actually does it for you
it's really hard to say like what files
should typescript tell you is broken
because typescript itself doesn't have
the knowledge of the bundler because you
have to know where your entry point is
what's running right like there's like a
lot to it and so you'll get these kind
of weird experiences typescript is great
it's much better than than that than the
alternative but it's also difficult you
don't get the whole thing like go is
also really great right you go you you
make a small mistake in go and it'll
tell you everywhere immediately in like
zero in zero seconds it's great it's a
good experience typescript is at least a
second rate great experience which I
think is a a dramatic Improvement in
JavaScript land Stripes developers
productivity team aims to provide our
Engineers with the most productive
development environments of their
careers and Delight in our tools are
crucial for that we work hard to
identify the most pressing issues
affecting Developers for example we've
built Integrations into all of our
development tools for reporting friction
oh that's kind of cool I think this is
really cool I love the mentality if you
are a company of three people this is
not important if you are a company of 30
people 50 people 100 people this is so
important the care for internal tools is
just so hard to sell but it is such a
multiplier in productivity you just have
to like LSPs are not hard to create lean
into it you have some proprietary
configurations build an LSP link out to
documentation make it easy for people
just to use you internally right like
just just go all in on it and so like
for me it's just it it's just all about
that which is how how well can you build
internal tools and so I just love this I
absolutely love that they're taking it
most importantly it Mak Stripes sound
like a cool place to work honestly which
is quickly routed to the responsibility
teams and prioritize typescript support
was one such pressing issues and team
supporting front-end Engineers began to
plan out supporting typescript across
the company choosing the right migration
strategy which was all of it at once our
largest frontend codebase Powers the
stripe dashboard and other user facing
products the dashboard codebase has
tight coupling between disperate
components and no cleanly factored
dependency graph an incremental
migration to typescript would force
developers to work in both languages to
accomplish common tasks we would also
need interoperability layer to sync
between definitions between both
languages and keep them consistent
throughout the development process
that's a yikes from me Prime got a
different definition of friction he's
moving so lightning fast that friction
going to friction I'm not even sure what
that means to tell you the truth in late
2020 we formed a new horizontal
JavaScript infrastructure team a group
of Engineers solely focused on elevating
the experience of writing JS at stripe
one of the team's first challenges was
to replace flow with typescript with a
long and uncertain migration we began
speaking to companies who had ran into
similar migrations and reading articles
from Air table and zappier describing
their experiences these companies
developed automated scripts to convert
one language to another ran them over
their entire code baces and merg that
output as a single commit air table had
published their conversion script to
GitHub as a source to Source conversion
tool or code mod that would parse flow I
mean this is super cool I absolutely
love this I did a like a semi similar
experience I don't need it anymore but I
had this really annoying problem at
Netflix where I had this untyped
database database by the way and
it would just send down just gallons
gallons of just Json objects so I made
something called undefined and which
would take all of the objects I'd feed
it like like 3 million objects and then
it would like Union and collapse and
create and then you could start naming
it in config each one of the names or
else it just give you like event one
event two event three or whatever you
wanted to call it and it would just
build out this entire definition based
on a huge amount of just untyped Json
very fun it was a fun project but
generating types is hilariously fun can
be great but man is it also a pain in
the ass I can't imagine flow to
typescript I'm sure there's these weird
edge cases that are super annoying
migrating in this way would greatly
reduce the cognitive overhead for
Engineers who would not need to handle
both type systems for the same product
Behavior we could have a clean break
between flow and type script imagine the
next day like after you put this in
imagine what would go and how many bugs
you'd have to deal with over time so
you're saying you built quick type I
don't even know what quick type is but
something like that only Prime would
name a project undefined yeah it was me
saying I hate you because the best part
is is that when you required in you have
to require in undefined super funny we
are really impressed by the quality of
air Tes conversion code and decided to
use that as a basis for our migration
efforts many thanks to the the team at
air table for building this out and
sharing their work the Open Source
Community benefits a ton from examples
like this we began by copying air
table's code mod to Stripes mono repo to
run against our internal code our
JavaScript projects make heavy use of
sale a shared design system of strictly
typed react components I don't know what
this means so that was our initial area
of focus we generated typescript
definitions for sale rather than
converting the code to typescript it
would continue to support applications
written in flow to safely support both
type systems we wrote tests to verify
that typescript definitions against any
changes to underlying flow code this
approach would be too cumbersome for a
large code base but thankfully the sale
component interface is explicit and
quite rigid from the outside this just
seems crazy the core code model was
solid but not comprehensive for many
files it would crash to generate
imperfect output over several months we
iterated to handle most syntactic and
semantic edge cases for one example
JavaScript Arrow functions can return a
single expression without a return
statement such as the following y
JavaScript object literals uses braces
to wrap the definitions because those
braces are used to delineate blocks of
statements returning an object from an
arrow function requires an additional
are they not using some sort of like
parsing tool apra or whatever people use
these days we noticed that code mod was
incorrectly stripping the extra
parentheses from these Arrow functions
but only in the case of generic
functions a function that takes a type
argument which is syntax not available
in standard JavaScript by the way I
would say the bad isn't that it's not
being handled properly it's the fact
that you're like people got to stop I am
perplexed at the usage of Arrow
functions I am okay I'm perplexed by the
use of Arrow functions and c and cons
like this I guess I just don't
understand it it's not it's not even
like it's smaller right it's not like
function
Fu it's not like you're saving it's not
like you're saving code you're not doing
anything more glorious here like what
what are you buying here what are you
what are you buying here yeah it only I
mean if you're doing this I would just I
would have so many questions the
chainer's like well it makes sense if
you're doing something like array map
right like this actually that makes
sense right we can all agree okay this
is a good use of an arrow function but
you're defining it like a function
that's what I don't understand is like
you are clearly creating a function I
just don't get it hoisting no one relies
on hoisting none of that makes any sense
standard languages for the last forever
other like every language other than c
when you define a function X it is
expected to be within the scope of X
like that's just an expectation so you
call X here you can call X here like
this is just like it's in Rust it's in
other languages people just expect these
things to exist in fact it's super
annoying when you have to Define your
functions in order making it named
function gives it better error message
it does it's nice people love ligatures
right I don't understand why like I get
it for you know you're just having a
small Anonymous function or you're
actually using this like there's a real
reason this it makes sense I just don't
understand it as like a top level
function it's just like just yo bro just
write function you're not you're not
buying anything once sale was unusable
from typescript we worked on a couple
internal applications containing
hundreds of JS modules we also added a
second pass to code bond to suppress
errors in the generated code using
typescripts TS expect error comment to
tag these errors rather than resolving
every error ahead of time we focused on
eliminating flow as soon as possible
tracking typescript error suppression to
address the conversion after initial
pass on the dashboard codebase created
over 97,000 error suppressions with our
interative approach updating the code
mod we are able to get that number down
to 37,000 or about one per thousand
lines of code 3 7 million lines of code
did anybody else catch the 37 million
lines of code what the hell are you
building you'd have to type one you'd
have to type a line of code a second for
a year and a half to get that much a
th000 time a th000 is a million so 37
1000s would be 37 million just add three
zeros dog just add three zeros right
here you don't obviously they don't
count node modules because node modules
aren't in flow man this is comp dude
Rick rich clearly is here rich and Brad
and Tom combined made 37 million that's
just c i I'm having a hard time
fathoming 37 million any anything okay
like that's hard for me just to even
think about both flow and typescript
support measuring type coverage and we
are pleasantly surprised that typescript
reported higher coverage than flow even
with the suppressions we attributed that
to an increase in the number of quality
or third-party type definitions
available in typescript the lack of
which was a large contributor to poort
now I got another question about
JavaScript type coverage here's like a
no no of JavaScript that you may not
know const fu fu equals Json oopsies
what is it gu right Json parse you know
some string right remember what comes
out of parse is an any so it's like do
you have code coverage here do you have
type coverage I don't think you have
type coverage you have type assumptions
so you have to have something like Zod
right you have to have something as Zod
a lot of people don't realize how tricky
typescript really is cuz typescripts not
tricky when you have the things defined
typescript is tricky at every place in
which you have some sort of contact with
some other world or where in any can
exist that is why we have ambient
declarations to make Json pars return
unknown yes I think that's Matt did that
right he did that whole like fixed
typescript thing that was a very good I
think that was a very good move TS is
great because it can give you false
sense of security TS reset that's what
it is as we moved our dashboard with its
tens of thousands of modules our
approach created significant memory
pressure on the typescript compiler our
primary tool to address this was
typescript Project references although
the dashboard is not structured as
distinct modules we could infer a module
structure and create project references
based on that this approach gave us a
Headroom to run typescript over a code
base without refactoring large chunks of
application code all this makes perfect
sense I don't think you're going to get
away from the memory problem at 37
million lines of code like typescript is
going to use 20 30 40 50 gigabyt of
memory I don't see how you're getting
away from it you have too much code and
typescript and JavaScript aren't light
on memory so it's just super hard it's
3.7 it's a typo how do you know it's a
typo I you know what I'm going to I'm
going to I'm going to believe you it's a
typo oh it's in the typ oh nice you're
right it is somewhere near the title I'm
not sure where it is but I'm sure it
is you're right it's right there okay
cool but so you're right it is 3.7
million so it's a typo it must be one
per 100 lines of code but I still have a
question 3.7 million lines of
cod that's still huge are you're right
it's better it's still crazy all right
going live hundreds of Engineers
contributed to the dashboard each week
such a sweeping change would be
exceptionally challenged to merge in
normal working day so our team decided
to commit to a date March 6th a Sunday
where we' lock the stripe monor repo and
land our branch in the week before
merging we focused on passing a build
through our CI system and deploying it
to our QA environment although
typescript could successfully check the
project other tools that process our
source code es lint just webpack Metro
would also need updates one particular
pain point was just snapshot testing oh
really just generates snapshots with a
hard-coded reference to a test file that
generates them since our code mod would
generate either a TS or a TSX extension
for typescript files the snap snapshot
would have invalid references back to
their test sources we simplified by
switching the generated to only use TSX
this meant we could write the snapshot
in bulk and keep 100% of those tests
passing I mean whenever you do this kind
of stuff this is what you're getting by
the way in some cases we recognize that
fixing the code of our typescript
compellability would add weeks to our
schedule one set of cases was our custom
es lint rules we had a rule to reorder
Imports to enforce consistency between
files I hate this by the way I hate this
I honestly see no value in reordering
inputs alphabetically can someone help
me what what's the purpose helps people
with OCD you know this is called
exposure help right you slowly get first
off it's not OCD okay you're just being
an okay there's a difference
between OCD which is a real medical
disorder and requiring other people to
organize their Imports alphabetically
that's just called being an
there's just a big difference between
the two I don't know who's organizing
Imports I have never had any I I don't I
don't understand it Group by Imports by
type yeah I mean I've seen people do
this but again like I think all of this
doesn't make any sense any anymore let's
keep on going with passing buildin hand
we reached out to our product teams with
user facing functionality in the
dashboard although the dashboard has
extensive unit in functional testing it
has limited end to-end test coverage
this made manual testing by product
stakeholders crucial those tests
highlighted some minor bugs which we
resolved during the final week in one
case we are failing to load any
translation for non-english dashboard
users due to a hard-coded JS extension
in the translation Landing code damn
that sucks funny but that sucks this
process gave us high confidence but
there's always uncertainty with the
change this large although we had a fir
grasp on our developer tooling and build
processes we were mutating every file in
the codebase subtle errors in our
conversion scripts for example removing
an empty field from an object shared
between multiple components I mean I
think it's interesting uh I mean I don't
know if you've ever done any sort of
migration it's it's really really hard
to migrate large amounts of code and
this is clear on Saturday March 5th the
team generated a new migration branch
and ran automated scripts when we
deployed that Branch to QA and repeated
our validation process and including
manual testing suggested by products
team we found no issues man such a good
feeling people tell me that's not a good
feeling thanks to our Caren rigger of
the previous year's work we had no
unpleasant surprises as we shifted
traffic to the new code we unlocked the
repository and let developers know the
dashboard was now in typescript Crazy
imagine the first try imagine the first
try on that that is awesome good job
team the conversion was not perfect over
the subsequent weeks the JS infer team
addressed issues as they arose natural
one example we didn't anticipate was
Engineers reporting inconsistency
between CI and local typescript runs yep
makes sense typescripts we are able to
use many third-party type definitions
installed from npm and if those are
updated Engineers will need to install
new versions Yep this was different from
our flow configuration where
dependencies updates rarely changed
types so we had to educate Engineers
about running yarn and stall as a
just I just I just love the fact
I I just love it I just love it I love
the fact that we live in a world in
which our interpreted code you need to
delete it and reinstall it regularly to
ensure that you're having a good
experience like there there's something
about that just just tickles my fancy
beyond anything else there's still more
work to be done we know performance
improvements further with our granular
project references and better caching
could speed up our CI runs however the
benefits have far outweighed the bumps
along the road Engineers enjoy features
such as automatic dependency Imports and
code completion as well as typescripts
communities extensive Corpus of third
already type definitions and
Integrations yeah these are all great
things having an LSP is a good thing can
we all agree having LSPs is a good thing
when new Engineers join stripe to write
front end code from day one they can be
successful in a language which they are
more likely to be comfortable and
familiar I really hate this statement
which is I think the reason why we have
so much JavaScript to begin with if you
call yourself an engineer and yet you
don't feel comfortable switching
languages you're not an you're not a
you're not an engineer if you're an
engineer you you're solving problems if
someone said I had to use Objective C
which I did have this happened at
Netflix I wrote a multi-threaded client
in Objective C my first time ever that's
just life that's called being an
engineer you have to just simply do what
you got to do now if you just want to be
a react Andy go be a react Andy no one's
going to hey no live your best life okay
if that's what you want go do it first
try literally first try and it ran on it
ran on iOS for six years 7 years wrote
it in like a month and a half it was a
ton of fun I had a ton of fun doing
Objective C because I just made it fun
as opposed to making it shitty sometimes
you just got to do what you got to do
anything but Java I've had to write a
lot of java I was the first person to
spearhead metric reporting at Netflix we
had this problem where we had all of
these images for all of our Originals
coming out and you have all the images
localized to every language showing up
in production and we didn't realize
which ones were and were not so what did
I do I took that system that we already
had for fetching images and I started
putting in a bunch of reporting and I
just created an entire reporting I
called it Atlas mantis logger or AML it
was monad likee it was monatic so you'd
keep on it was pretty fun had this like
fun little approach we did to it but it
would allow for Atlas logging which is
like this persistent counter gauge graph
and then mantis logging which is real
time so you could actually just log on
put in your like computer and it would
just flow through and show you what's
exactly happening in real time as you
scroll through the website to go okay
this is where we're failing or it would
go into a persistent long-term one so we
could actually say hey images are
identified as being missing in these
languages for this video IDE go make
sure that they're there boom right and
that was all in that was all in Java
that was fun I had a lot of fun doing in
Java too what's a monad a monad is a
burrito when we shared our story of our
migration publicly the response was
equally enthusiastic developers from
across the industry reached out to learn
more and apply the same improvements in
their own code bases to support these
developers who were sharing our
typescript conversion code on GitHub
nice nice aside from the particulars of
JavaScript or flow or typescript our big
lesson from this migration is that
dramatic improvements to large codebases
are are possible with diligence
commitment and optimism we will apply
that mindset to other opportunities make
our Engineers more effective and hope
others do the same nice I like it this
is a great article this is a great
article and I think there's a lot of
really good lessons here that I I I
genuinely like I love the idea of of
going for it making large changes trying
to make people's lives better I love the
idea of having internal tools as a first
class citizen as like a this must be
done this is in incredibly important I
just think that that is such good
companies I get burrito comparison yeah
you know you got your outside and your
inside you know what I mean I don't know
everyone keeps calling monads burritos I
don't understand it but anyways I like
the horror stories instead yeah horror
stories are good but horror stories you
learn just as much from as good stories
this one had a really good meaning and a
really good takeaway which is we really
should take internal tools and internal
tool teams as seriously as any product
facing team because any internal team
that has a lot of support will produce
tools that create user-facing teams to
be sign significantly more productive
and at the end of the day that is what
you're concerned about reliability and
features the name is the primagen