okay hello good
morning uh my name is miow and uh we're
going to do some presentation about
Formula One and machine learning so uh
my title of this talk is predicting F1
strategy using ml.net or uh miow tries
to predict stuff that teams can't do
very well without the data that they
have and I don't don't have really a
journey so uh my name is miow I come
from here this is Northern Poland this
is a very beautiful picture that covers
both Poland and Southern Sweden and
bunch of Denmark although under clouds
so around decade ago I moved to to
Southern Sweden and work for a company
called Tre 37 we're a consultancy
company focus on uh knowledge sharing
and we love to share knowledge with our
community and our clients we've been to
n DC for over 10 years now and we're
very happy to present in front of you
today me today and yesterday our CTO who
a Renown expert on Disney
Princesses um okay so just to manage
expectations so if you want you can
still leave today you learn just a tiny
bit about machine learning a bit about
ml.net and more than you ever wanted
about
rubber um so yeah let's start with
ml.net
uh why do I even use ml.net since there
are so many better tools for machine
learning right like ml like net is not
really known to be like Powerhouse in
this area at least maybe not yet so
first reason is doet I I come from this
U from this kind of technology I did
most of my work in in net and c and
maybe a bit of FP so when I want to like
crack a bit of code my my natural
instinct is just like open VSS or vs
code and just write some C phone to like
test something out and
um uh yeah Microsoft done a lot of good
things in recent years about like open
sourcing stuff and accepting
contribution and uh uh this product is
run really well in that regard and um
it's pretty good API it's focused on uh
problems you want to solve and not kind
of kind of what kind of RIS you want to
multiply to you what kind of network
architecture you want to do and stuff
like that it doesn't really
support uh deep neural networks that
well but we'll get into that later and
um yeah it's pretty regular updated this
year was uh version two uh we we had
release of version two and uh yeah this
is often my first step if I want to like
you know look at some problem and uh and
see if uh if this is something I can you
know run some ML on this uh
uh you've probably seen already few
presentation about machine learning and
generally the the machine learning job
is divided in two parts first you kind
of create your model and it does the
longl lasting job first like training T
takes usually much longer time but first
you have to like prepare the data
reshape it and so on Pipeline uh so
ml.net has some like tools for that and
the other part is like using that data
to make some predictions to using uh to
creating some analysis so so like
inference uh and that's usually much
quicker job uh and you usually want to
do it like in separate task uh somewhere
else so when you train the model you
save it somewhere and you use somewhere
else you probably have some like regular
running job to train your model every
week month or something like that and
you're web app is like using that model
every second to make some predictions
and if if you're really stubborn you can
put like ml.net code like whole model
training and usage in one screen of
PowerPoint um but probably you don't
want to do it because first of all you
want to divide training from uh from
inference and so on but just for the
sake of uh showing how little um code
you need that's some example although
since version two 2.0 the API slightly
changed uh the old one still works but
uh the new one gives you a bit more
power uh so sometimes you want to move
in the
way um so let's talk about this my this
like pet project of my what was my
approach to this so I started with u
GitHub repository like machine learning
samples from uh ml.net this is like
really cool repository I I link it later
on slack so you don't have to look for
it right now uh where you can see like
different use cases for example like I
think the the on screen is like bike
sharing prediction is like regression
problem there are some like uh
categorizations problem image
recognition uh language analysis and so
on so you can start with those or find
like a problem that's similar to those
and just use that code kind of like a
starting point uh it's easier than just
type from
scratch and uh the the project plan in
like early 21 for this project was like
okay so like I'm going to start from the
sample and with like every race which
happens like during season every week or
two I'll gather data from that race so
let's say first race is usually bahin
I'll get that race and maybe from the
same race from previous years to have
like some repetition and maybe start
looking for some
patterns uh and then do some exploration
of data look at uh what's uh what's on
the offer in terms of tooling how can I
like visualize the data how I can do
like some automated uh retraining and
like ml
operations and uh yeah eventually go
from like one file script or like one
PowerPoint uh page to something that's
going to be nice well um well
architected
up uh so let's talk about formula one
strategy and since Formula One bond in
race refueling in 2010 I believe it
boils down to tires like race strategy
is bearing on the right tire in the
right time of the race um so this like
graph you can usually see after the race
which basically is uh on which position
were each driver was during the race and
you can see some like sharp drops for
example London Norris the top orange
line has some sharp drops here and sharp
drops here this is where he goes to the
pit stop to like change his tires uh and
when while he's doing this some other
drivers overtake him so he lose position
but there's benefit to changing tires
because on the new tire getting go
faster and apart from benefit he also
has to do it at least once because
Formula One and force to do one Pit Stop
um so there's like few ways you can do
it you can stop once and um then you
have like two kind of faces we call them
stins so this is like one stop strategy
with two stins and uh somebody else can
do like two- stop strategy so we have
like three kind of phases of race but
because of uh that that you know that
person changed those tires uh more often
they can you drive more aggressively
over Corners go faster they can usually
use use like softer tires which gives
them some uh benefits so even though
they spend more time in like pit to like
changing the tires eventually they end
up on top at the end of the race if they
manage to
overtake and uh sometimes something like
that maybe happen that like you have
really slow stop although we probably
should switch those colors because
that's usually Ferrari that messes up
the
stops and
um and uh sometimes uh you know we have
like situations like there was accident
on the truck there's safety car so all
the others cars go slower and maybe for
you it's a perfect moment to like change
your tires and instead of like losing 23
seconds you lose 16 so suddenly you know
you gain like 7 seconds with maybe like
three positions on the track and this
may be like decision between winning and
losing the
race um so initially I focused on period
from 2017 and to to 2021 it was like
good reasons for that it was like tires
used in that period were like pretty
much the same there were like minor
changes P Pally is always messing a bit
with those but uh generally like in in
architecture they were the same and the
AER aerodynamics um regulations was like
pretty stable during that period the
cars were mostly the same size same
weight and uh we had stable Core group
of like teams so we don't have any teams
dropping up and joining at least from
the core group so we have like data to
compare compare over the years which uh
is always useful and um 2021 was like
the last year of those regulations with
the new regulations came in 2022 so it
was like pretty good time to like play
with this and maybe like do like you
know good summary of that era but like
I've made that slide in 2021 and if you
look at
calendar you like realize we may have a
problem with the timeline of that
project um but yeah let's let's talk
Rubber and um I'm going to start with
this really confusing image from
Wikipedia but uh let's remove the wet
tires straight away when we have like
wet races the chaos occurs some of you
may have seen last week and uh there's
like no way to predict that it's
probably pretty stressful moment for
like team
strategist um to do and it's it's
impossible to predict what's going to
happen so let's focus on like dry tires
and on the left you have like compounds
t p designed five compounds although
there's like slight asterisk for this
year for that but um let's focus on this
uh C1 is the hardest C5 is the softest
but to not confuse uh people or maybe
confus even more they decide to like
label them with three colors so for each
are going to Peak three out of those
five tires and one of going to be the
the softest one of going to be medium
and one is going to be the hardest so
for example for Bahan in 2021 they
decided that C4 so it's going to be the
softest C3 medium and C2 the hardest so
they kind of pick the middle of the pack
and for the spine Spain of that year
they decided it's going to be more hard
compounds
um it's because like it's a usually very
hot race so the Tire degradation is
higher so they want to move them to like
a bit higher bahin is usually over the
night so even though it's like Middle
East it's not that hot as you think
initially and yeah at the end of the
race you can like see a chart like that
which drivers use witch Tires over over
time and I decided okay I'm going to try
to generate some data out of this uh
unfortunately manually but we'll get
into this and the kind of desired output
of this project be like set of
strategies for each drivers or team like
what be like fastest or maybe like
second fastest option for them to to run
but since we want to like minimize the
the scope of the project like run of the
MVP first let's try to like predict how
many laps we can do on like one type of
Tire let's say how many laps can Alex
Alburn do on a soft tire in Spain uh
with you know in the de like that okay
so we have first round of uh year 2021
Bahrain fantastic race fantastic track
and uh I manually generated a bunch of
data it's not much it's around 230
rows and um in machine learning the
problem when we want to like create a
function that takes input of some values
and generate one value is called
regression problem we basically try to
like match function to the shape of data
we have and uh okay so I like take
ml.net and just start to like setting up
like um what we need for a regression
algorithm and there's a lot of words
category there because a lot of data I
have as my input is not numerical data
it's just names for example driver name
is category we have like 20 drivers and
um well for regression categories is bad
vibes you don't want a lot of
categorical data
because when you when you calculate a
you know mathematical function you want
to like to Output uh numbers you want to
like multiply some numbers by some
numbers we basically generate like
polinomial
function and uh that's all fun in games
if you have like you know temperature
which is a number or like you know uh
humidity or stuff like that it doesn't
work like that when you try to like
multiply Max versus Sten by 0.37 right
it would work in JavaScript Pro probably
it what with something but uh I have no
idea what it would do in math so we have
to like do something called like one hot
encoding so instead of like having like
one category driver we create 20
categories is vers Sten is perz is
Ricardo and they have like varus Z 01 so
yes we have now numerical data which is
good but suddenly you have like instead
of one parameter we have 20 which means
for a you know mathematical part we have
20 more Dimensions to calculate which
means for this algorithm to convert we
would need to have more rows and I don't
have many rows right now so it's like
it's not very good situation categories
are not good for regression we're not in
a good spot with that data but okay
let's play with it let's see what we get
output I get some outputs and I you can
like compare line by line but it's it's
hard to tell something things look good
something looks bad but it's it's not so
good so let's look into statistic stcs
and you can like output from ml.net the
the model statistics on your on your
kind of comparison set and um loss
function is just kind of like a relative
thing to compare during training but R
squ is something more like more like
good uh you know non-relative value to
compare how good your model is usually
it's some sit somewhere between zero and
one and you kind of want this closer to
one and if it's
negative it's uh something like
this so yeah first race
uh but yeah the next race the next week
teams come up with new inventions I come
up with new models so let's bring up
some updates for imol and P Imon that
year and this is again not very good
situation because that was 2021 we still
have like coid raging around the world
so Formula 1 didn't travel to the usual
tracks so they other a bunch of like
random not very commonly used European
tracks like uh Imola and P imal Imola
for example is like legendary track and
it's uh you know it's a sacred land for
like Formula One uh that's where Senna
gave his life to the sport uh but
unfortunately it's very outdated track
it's not great for racing it it's narrow
at some places and formula one doesn't
like to come back there but because they
were forced they went to and ptim is
truck that's I think mostly used for
Moto GP but uh you know beggar are not
choosers so we we went with this so I
can add this data but it doesn't add a
lot of value because it's going to be
like I don't have a lot of repetitions
for that tracks um but yeah I added
around 200 rows of data 400 totally and
I started doing also some filtering with
ml.net because like if the stint
finished after one lap because there was
crash it's probably not very good St for
me to use in the training right I want
to just kind use successful STS let's
call them like that so I started looking
to like which ended with like Pit Stop
which ended with uh uh finish of the
race and did some uh filtering that
forced me to like learn something new
how you do filtering in uh in ml.net all
all nice and fun we did some
results uh some of them were not very
good because that was wet race so yeah
no point of like predicting the other
one like some things look very close but
again let's look at statistics and uh
yeah okay so um let's look at Spanish
Grand Prix and this is interesting
tracks because uh Formula 1 used to
tester a lot the teams know this track
very well the drivers know it very well
which usually renders hopefully like
boring uh racing because teams are
pretty much like very well prepared for
this and there's like no prises and um
let's to give you one example kimy
reikon and over his like 20 years career
spent like 88% of his Labs during
testing on this track and only like 12
in the you know
racing and um
yeah uh yeah we have a lot of new data
you know and started introducing outl
which is um you know there are a bunch
of regression algorith available in a uh
uh ml.net and you can just try one by
one to see how they do with your data or
you can even write a for for Loop you
know go over them to see how you're
doing with your data or you can just
like do atml experiment that will do it
for you and if you even that lazy you
can just you know Open Your solution
right click add machine learning load
your data and congratulations you're
data
scientist yeah training generating code
done ship it um I don't recommend that
but you know maybe it's a good start um
but something interesting happened
because we on the plus side right it's
still miserable but uh okay let's look
for more data and at that point I was
pretty like pretty annoyed with like
manually generating that data I was just
falling behind the schedule I was trying
to block that along the way and so on
it's just like
uh I want to I have like better ways to
spend my weekend right and then
um then I remember that there's this
library because there's like this weird
signal ER uh API that's like is posting
uh racing data during the race and you
can just like tap into this and listen
during the race and there's like some
group of people just like cut you know
uh take it over and like cash it and you
can just you know know pink. API and uh
it's written in Python but like it's
pretty easy to use it has like charting
built in so you can you know draw nice
charts of the stuff and so on so all
that time I was just generally manting
data and you know one evening on writing
Python and I have suddenly like 4,000
rows of data from all those years so I
don't have to like you know programming
programming
rules um so yeah let's say we're still
still in
2021 I want to predict uh zandor race we
had zandor uh just last week but uh um I
think that was the first year that uh
that race came back so it was um you
know I wouldn't like there was not much
data to predict on top of that and U
still somehow this algorithm converted
to something uh it's still not very good
result but it's significantly better
than uh than what we had before right so
we're heading in the right direction so
now it's 2023 and uh what's the reality
of this project so far uh I started with
samples that was easy and uh I stopped
pretty quickly generating the data but I
found another way so okay that's a win
and um I didn't do any of that because
I'm still working on like three scripts
doing this stuff of of course not in the
uh well architected up and year 2022
happened so we had like very big changes
in Formula One and uh one of them is
like the biggest change in regulation of
aerodynamics for like three
decades uh a lot of change to help uh
you know cars follow better behind each
other and also tires change Formula One
moved from like I think 13 in to 16 or
17 in Wheels which makes the tire all
significantly stiffer and those tires
act completely differently so like all
that data is probably
useless but maybe it's worth checking so
right now I have uh
around I don't know 6,000 rows of data
total and over 2,000 of them are like
already past those CH regation so I
started running those models like
including both including only one and so
on and at this point point it's like
inconsistent I'm not sure but probably
just going to tend to move towards the
newer data because that's what uh what
makes
sense um and there are a few problems to
solve like you know the the categories
problem that uh I mentioned that's uh
it's not very useful so one of the
categories is track and there are like
22 tracks a year so it's another
explosion of Dimensions but every week
or every race when pelli publishes their
you know the the tires they going to
suggest for the race they also uh
publish some information about track
that are some numeric information so
maybe you know we could use some of that
check how they're going to be useful so
instead of like using Monza as a
category I could just have like two or
three numerical values of those do that
make sense that would probably help a
lot so what's happening right now I
think the something I I train also tried
to train it yesterday evening but didn't
finish it up is try to model model lap
time
over over like kind of lifetime of Tire
because if you like predict that you
know Alex albon can do like 23 Laps on
the soft
tire this is somewhat useful but you
don't know how much you can push it if
you can do like 27 Laps on that tire
does it make sense because the tire act
weird they're like black magic they
usually like they start pretty much like
under performing the start because
they're not heated up to the like
operating temperature and when drivers
start like driving them they they get
like heated up and they like for some
time operate in like Optimal thing but
something starts to happen uh at some
point and that window is probably where
you want to like pit because after that
it's what drivers call Cliff so the
tires instead of like you know losing
half a second on the lap compared to
like your opponents you start losing
like seven uh you know seconds on the
lap so um
you know maybe at at the left side of
that blue field you think like okay I'm
losing like 2 point like I don't know
one second overlap and uh was going to
waste like 20 second on the pit stop I
would be catching up 20 seconds for that
but if you're like on the rle of H you
start losing s seconds on the lap that's
only like three laps to catch up so you
know that's like significant
consideration uh when you want to tip P
pit and like how much you can push those
tires sometimes you can hear over the
radio U drivers saying like uh you know
how do you what do you think about like
plan plus 10 because they want to
stretch that that tire if the conditions
change so that's what's happening and
another thing is like Pit Stop times
because as we know some teams are not
really good at this game and some are
really good and you know Pit Stop uh
from the lead to just get extra points
just for the for the fun of it and um
that's from that point it's pretty much
like you know packing Rock sack with
those information how many time you take
on the pit stop how many things it's
basic optimization problem another thing
I saw people doing with that API is just
using GPS data with some like calculus
black magic to like calculate lateral
forces on the cars and tires and so on
and basically doing like nearly 3D
animations of cars going on the track
it's amazing from just amount of data
that comes from here um and that could
be like something pretty fun too but we
have to like deep into like black magic
uh so yeah there is GitHub project with
that stuff I need to push that python
script so if you want you can generate
the data I
generated and uh that was a bit speedrun
of this presentation but uh um we can
take
questions or
not all right then uh I'm I'm going to
be here rest of the day feel free to
talk to me later and oh
yeah okay can you repeat
sorry yeah right now I do like uh
categorization and one hot en coding but
it's like not very good thing and uh
that's uh I had this like Pally slot uh
that's what I want to start using is
basically use some like you know
information that okay instead of like
using Monza as a category you say okay
this is a track with like really high
high degradation with like bunch of fast
corners and so on uh to you know to have
some numerical data instead of like Cal
data that's uh what I'm aiming at but um
I don't know haven't experimented with
that that much
it
all right uh see you outside thank
[Applause]
you