cool
okay let's start
welcome everyone hope you had a
good good lunch um
what i'm going to talk about um today is
[Music]
our async code reviews and
the study that i did
that has spanned a
period of six plus months
at the end it involved typical product
development teams
uh i analyzed around 40 plus very active
repositories and tens of thousands of
pull requests so
today i'm going to
share with you
the conclusions that i got out of the
study
so my name is dragan stepanovic i work
as a senior principal engineer at the
company called
talbot which is a deliberate hero
company
and
my main areas of interests are xp
extreme programming
poc
which is theory of constraints lean
systems thinking
and i tend to rant on
social media and
blog
and there's also one terrifying fact
about me is that i put my own pizza for
some reason
i don't know if there are any italians
in the audience but yeah it'd be really
funny to see their faces now
so just to frame the talk for today
i'm not saying that the context that i
had is the same context that you had
um i won't be telling you that you
should do the things that i concluded
for for the teams that i've been working
with
um i'll just share the
learnings that they had out of this
journey
so
not sure how many of you have heard for
this quote
nobody ever got fired for buying ibm
and the quote comes from 80s as far as i
remember
and the idea was that most of the
folks from the procurement departments
went with ibm as a default as a vendor
because ibm at the time had a huge
market share
and it was easier to go with them
because everyone else went with ibm and
um
if anything goes wrong it's ibm hey with
the top
and the other thing is that
the folks that wanted to go with some
other vendor
had to answer a lot of questions that
made made the whole process expensive
for them to procure someone else
so it really depends on the context
in which you're operating in
and what are the problems you're trying
to solve
so when i say nobody ever got fired for
by buying ibm
and when i think about the typical
software development process that we
tend to have in the teams
i often think about the pr based async
code review
so i try to visualize it here
and
we have let's say two developers emma
and luca
and it's a start of the sprint let's say
that team is working in a sprint in
scrum
methodology
and um
emma pulls in ticket number one here and
local pulls in ticket number two
so they start working on their own
respective tickets
and
at one point
emma
codes make some changes introduce some
changes to the system hopefully tests
and at one point she
figures out okay i think i should ask
for a review for my peers
so she creates a pull request
and invites luca for the review
but look he's busy with something else
so he's working on ticket number two
there are lots of other things going
around
um lunch
breaks
outside of the business hours
review meetings checkings like checking
email etc reviewing other pr's
so luke is not able to immediately
respond
and what happens is also emma while
she's figuring out okay since i'm
waiting
i want to be a good employee so i want
to keep working on something so she
pulls in another ticket take number
three
at one point she figures out about
is not
answering maybe i should uh remind him
and
i often see these plays and you know
these begging emojis in this kind of
systems where the working progress is
really high
i would even say that the amount of
praying hands that you see in the
systems is proportional to the amount of
whip that you get to see
so the system is telling you something
but at one point luca is able to come
around and he reviews the change asks
for some um reviews the pr asks for some
changes
um and sends back the pr to emma
but them is busy with something else now
she's working ticket number three there
are lots of other things going on so
she's not able to react immediately now
and eventually she comes around and she
incorporates the changes
and they go back and forth
and at one point
they converge approve the polarquest
emergent
so interesting thing to look at here is
there are lots of things going on but if
we focus on ticket number one
and we look at its lead time from start
to finish there are two different
types of things that are going on with
it so one thing is
ticket number one is being actively
worked on
which is called in lean processing or
touch time
and the other way other thing that is
going on is the wait time so
when the
item is just sitting there in a queue
waiting for someone's attention
so to provide a bit more of a context of
the study that i did
um i tend to advise
and
mentor and coach teams
um
in a way that we try together to figure
out how to maximize the throughput of
the value of the whole team
and one of the things that they tend to
use but to look at is the
way that people work in a team right
so
my guiding principle is meet people
where they are
and often in the teams that do
peer-based async code reviews
there's a certain
percentage of
managers and engineers that would like
to have a bit more data informed way
when it comes to different way of
working yes we would like to try out
something else but how does it compare
to what we have currently now
so we would like also to visual or
visualize our current system
so what was i curious to see
me coming from the extreme programming
background which is
very heavy on collaboration as you might
know peer mode programming
whole team approach
i had a hunch for a couple of things
that i was really curious to see
one thing was engagement
and
the
effects of the delays that we get to see
in the system and because of this high
working progress
the other thing was wait time
um going back to this visual that they
just presented
and trying to figure out how much of a
wait time we get to see in this kind of
systems
and also how does that compare to the
size of the pull requests or the amount
of effort that we put
in in
creating
work items
so going into engagement
why was i curious about the engagement
so
i had this analogy in my head
if i have a phone call with someone
and there's a delay in communication
between us
what tends to happen is that the
conversation dies off very soon
so
because of a delay the engagement has to
go down and because of that the
communication ties off meaning
we
faster converge
and
the other thing while i was what i was
curious about when it comes to
engagement was uh the choked feedback
and when i say choked feedback
i refer to
the usual solutions that we use for
reviewing public quests providing
feedback and comments which is you know
github gitlab etc and the way that most
of the teams provide feedback is in a
written form
and written word used way more expensive
than
verbal communication takes a longer time
to write
and it's also because it's delayed there
are other problems with it
and i notice there's
way more feedback provided when it's a
verbal timely feedback when it comes to
the
collaboration in the team
so that's what they call the high
latency low throughput feedback
in this kind of mode in this way of
working
and
so going into the first scatter plot but
this is just a sample i think this was
500 public fest
um each dot that you see here is a
single pull request
so
on the x axis you can see size in lines
of code now when it comes to the
complexity of the work
you can measure it in dozens of ways
one way to is to just go ahead with
lines of code
the other one is to i don't know count
number of affected files of number of
packages etc but for the study that they
did i noticed that
simple lines of code was enough
and on the y-axis you can see the
engagement now the engagement the way
that they calculated it was that
i was
looking at the amount of feedback in
terms of the amount of comments that
they get to see
and
but i also had in mind that
lots of the comments um
were
trivial and
definition of trivial for me
was
kind of um i used to finish it it's kind
of silly but it's like
less than four words
and the idea behind that was that i get
to see often especially in the big prs
this lgtm plus one looks good to me etc
right so these are the
comments that don't really provide
actionable feedback for people to
act on it
so on the y-axis you can see the number
of control commands
now it's a question what do we see from
here
probably nothing right there's just a
cluster of data
um from this representable data set i
would say most of the prs are less than
500 lines of code
and have less than 12 nonchalant
comments right but then i started
thinking about it
and
if i invest let's say a lot of time
let's say seven days working on
something and i get certain amount of
feedback it's not the same as if i
invest 10 minutes and i get the same
amount of feedback right
so i was thinking what happens if we
actually normalize the y-axis the
engagement
will be the engagement
per size right
and plot it against the size so the
engagement per size is the number of
controller comments per 100 lines of
code that we can see here
and this was interesting but it was also
expected in a sense
so it was interesting because
what we can see from here is that
as you increase the size of the pull
request
the
amount of feedback that you get tends to
go exponentially uh down amount of
feedback per size right
so this was um
for me meant something so i was trying
to interpret this
and this was the insight that i got from
all of the data set that i've been
analyzing
and
the thing is if we use code reviews as a
process at least as a process to provide
feedback in order to build the quality
in
if we are not able to get the feedback
then we are also not able to build the
quality
so what i'm saying is that
most probably
this means that
as we increase the size of the of the
prs
the
there's there's a higher chance that we
have lack of engagement right
and that means that um we are less
likely to build the quality right so not
saying anything about the left-hand side
i'm not saying that you if you have
feedback that you have quality because
it really depends on the type of the
feedback security of people providing
the feedback etc right so i'm just
saying that if you're using this method
in order to build the quality and we're
not able to get the feedback in the
first place then we're not able to get
the quality as well to build the quality
and
this was kind of
expected systemic insight that was
expected for me
i love this quote never had a huge pr
that didn't look good to me
and
perhaps some of you also saw this um
this week
so um
that was expected in a sense right i
think all of us already have this by
laughs i can understand that we share
the experience
and when it comes to building peer
building quality and with big prs i just
recently bumped into this
tweet and this is how i see it
so reviewers feel like these viewers you
know spreading their arms trying to
prevent people from going onto the pitch
but it's not really happening
um
so that was engagement that was the
thing that i kind of expected right but
then when it comes to the wait time i
had some surprising insights that i also
didn't expect
so if you go back to this visual here
and focus on ticket number one and also
look at this
a timeline at the bottom
we just focus i think at number one and
see its lead time
we can notice that
there are parts of its lead time where
we are actively working on this item and
parts of its lead time when this item is
just sitting there in a queue waiting
for someone's attention right
so i was curious to see how does this
look like in
in terms of the of the numbers
and
the way that i went about
defining the wait time
is approximation so every approximation
is good in some cases in some cases it's
not
so there are some assumptions related to
it
um and if you're interested you're going
to find um
a series of articles where i dig
deeper
inside of of this metric
and trying to understand the uh list
down the assumptions and the
preconditions for the assumptions being
good enough but the way that i went
about it was
a typical developer taking a ticket and
they start working on an item
at one point they figure out they are
done they invite other people for
for feedback and at that point the wait
times starts to
tick so in the sense of
from the last commit that we get to see
or from raising public fest until the
public task has been merged
that's the way time approximation here
if i find a data set that tends to
indicate that there has been
significant or substantial amount of
back and forth in the sense of comments
being exchanged between reviewers
and author what i see
commits after raising public quests that
indicates that the approximation is not
good enough
and that's one thing to also have in
mind
so
going into
just one of the
examples
um so here we see um
just results for
um this was data set of 500 merch pull
requests
and uh
this uh team they they took them uh six
months to produce this uh 500 police
meaning that it took them 500 uh it took
them six months to push these 500 pull
requests through the system of work okay
and what was interesting to see is that
the wait time cumulative in in in months
was almost 28 months
now these numbers are not accurate
but if the even if they're accurate 30
percent they're accurate more than that
um
they're a really good conversation
starter so people start talking about it
in a sense of what are the things that
we can improve um based on this right
so
this is also important because
if we think about digital products and
generally software right we use it in
order to deliver value to our customers
now
if
the work is stuck in our system for a
longer time
that also means for the cases when we
deliver value to our customers we delay
delivery of the value to our customers
right for these things that have been
stuck for this time
i'm not saying that all work is equally
valuable there are some things that
are less valuable than the others but
they're all always kind of stakeholders
and everyone
every change usually has some impact
be developers or some other people
inside of the company
or
hopefully customers right
so
this is um the going into the scatter
plot for the wait time
on x axis we can see again size in lines
of code and why we can see wait time in
hours
so what we can see here is
not much again
most of the prs take less than 200
hours to
of a wait time right but then i was
thinking again
if i
work
let's say seven days
and i wait for um and raise a pull
request and then i wait for feedback for
two days it's not the same as when i
work 10 minutes and then wait for
feedback for two days right
so what happens if we plot the wait time
now to be wait time precise
so the things become quite
different
and this was also the systemic behavior
that was visible across
all the data sets where the teams have
been doing gays in code reviews
now i was thinking how to interpret this
and
the way that they understand it is that
as we reduce the size of the pull
request
the wait time per size goes
exponentially up meaning that
the cost of code review per size goes
exponentially upright
this is really interesting
because what we're saying is that
the smaller the change the more
expensive
the more expensive it is to push through
the system right
and
i have been advising the teams that have
been doing um big pull requests you know
just
reuse the size of the pulley request
make it smaller
but then i found out through this
study that actually small prs also have
trade-offs when it comes to the async
code reviews right and why do we want to
have small prs because
the idea of small batches from lean
smaller pr's are quicker to write they
are also
quicker to review
people need less time allocation to
review the public vests also coming back
and forth is
usually quicker they're less risky
because less change
introduced
when we talk about the accelerate and
for key metrics they
shorten the lead time to change and
increase the deployment frequency etc
etc
so
that's the way that they see the
the
the behavior that this kind of system
incentivizes with this kind of economics
right
and
thinking about it i'm sure that
lots of you have
came across
the cases when
let's say a team has a test route that
takes to takes a long time to run let's
say uh there's uh it takes them 20
minutes to run the test suit right
if that's the case then i'm not going to
be
running the test suite after every line
of code change right because it doesn't
make economic sense why
because most of the time i'll be
spending just sitting there waiting for
the tesla to finish right so what the
system and economics here in this system
is incentivizing me is to increase the
size of the of the batch right introdu
introduce more changes to the system
before running the tests
in order to get biggest bang for a buck
so the way that they see it is that
inventory in the system so the amount of
changes that you can use for in this
example
expands until
it matches the delays that you get in
the system
and
why is this important because
i don't know how many of you remember
this
[Music]
game that at least some of us have
played when we were kids
with siblings or cousins but
um
someone hides the item and then you know
the other person provides the feedback
but they provide them feedback
um you know is it
hot
warm or cold and based on that feedback
they're able to find the item right
now if you have two teams where one team
provides feedback after a minute
and the other team provides feedback
after a second
i guess that most of you are going to
bet on the team that that
gets the feedback after one second that
they're going to find the item sooner
right so there's the same idea about
why optimizing the process in this case
will lead you to um
to find a custom of value for a customer
sooner right because the learning
cadence is faster okay and um i love
this quote from don reinersen so while
you may ignore economics it won't ignore
you because of the incentives of the
system that have been set up
because of that
and
i was thinking about how to
you know how to map out this model
um that they haven't had in the sense of
why is this the case you know why the
teams that do ace encoder views also
when they try to reduce the size of the
peers usually end up going back
and
this is a close loop diagram
if anyone is familiar so with it it
comes from the assistant's thinking and
i'm just going to give you a short crash
course in it um it's quite simple so
there are
variables that we see here as labels
and variables can be
connected in two different ways so they
can be there can be either minus sign
between them
and there can be plus sign and this link
is a causal link so if i have one
variable and the other variable that are
connected if this one moves this one
also moves if the sign is plus if this
one moves up this other one also moves
up and the other way around if the
relation is minus meaning the opposite
when one variable goes up the other one
goes down and vice versa okay
so interesting thing also happens when
mapping close the loop diagrams is that
you might notice after connecting all
the things that some
that some feedback loops pop up right
and there are also two important
feedback loops
that are worth considering or actually
part of the causal diagrams rather
one is the reinforcing feedback loop and
the other one is a balancing feedback
loop now reinforcing feedback loop as
you might guess is this kind of snowball
effect
and the balancing feedback loop is the
feedback loop that
kind of seeks the target in a sense so
at home
you know you when you set up the let's
say thermostat to 21 degrees you know
the system is pushing the
um kind of it's working until it
achieves the target right so if you open
the door and outside is called it works
even more in order to be able to achieve
this target so this is a balancing
so let's just go uh quickly through i'll
just walk you through the the diagram
so here if we have a
peer size of certain size
pr sorry of certain size and we try to
call it let's say
if we reduce the size of the pull
request then the motivation or incentive
to review goes up because people want to
review smaller peers usually right
that means that also time waiting for a
review from author's perspective goes
down
which means that their perceived cost of
code review per line of code also goes
down
which on the other side is incentivizing
them to keep reducing the size of the
broadcast and that's a reinforcing
feedback that we get to see here which
is a great thing to have
now
there are other things also
going on and that is that if you work in
the async code reviews manner
and we reduce the size of the pulley
quests then the number of peers to
review in a given unit of time goes up
right so if it took me one week to
develop a pr
and let's say everyone on team
takes the same amount of time to develop
that
then we when we have the prs then we're
going to have twice as many uh pull
requests to review in
a week right
so the thing with a
higher number of
pull requests to review is that they
also mean
um higher number of interruptions for
reviewers and also for authors in a
given unit of time right
and if you keep reducing the size of the
pull request you're going to hit the
point where the number of interruptions
is unbearable for people and everyone
wants to protect also their personal
flow
and
that also means that
the motivational incentive to review
uh given pull requests goes down which
on the other hand
increases the time waiting for the
review from author perspective and the
perceived cost of code review goes up
which incentivizes the pr size to go up
so that's a balancing feedback loop that
is balancing out this reinforcing
behavior and there is a shift in
dominance usually between these two
loops as you keep
reducing the size of the pull request
balancing feedback loop tends to
gain more
um
dominance
but there are also thing other things
going on here that we saw with
emma
it's that
is that if we increase
the time waiting for a review while emma
is waiting for a review from luca she
starts working on something else right
and that increases the ups that
decreases the number of peers to review
right so it kind of reinforces this
balancing feedback loop that we see here
there's a problem here because we have a
conflict here so from one side
the system is increasing the
motivational center to review and from
the other side is decreasing
so
that leads me to a flow efficiency one
of the metric from the matrix from the
lean and um flow efficiency is kind of
simple metric we look at the lead time
of a given item
and we figure out when
we have been working on the item which
is a work um here or processing or touch
time
and
we
also look at how much
the item is waiting sitting there in a
queue
and we calculate
out of the whole lead time how much of
processing time was there right so the
higher the flow efficiency the better
the process you have and the opposite
so i started analyzing flow efficiency
for these data sets
and what i got to find is this um kind
of a behavior
um so on the y-axis you can see the the
flow efficiency
and
here
in this data set the flow efficiency
starts plummeting around 120 lines of
code doesn't really matter
but
the important thing or important point
to have to think about here is that in
this kind of system with this kind of
economic
let's say that we want to introduce the
300 lines of code change right so and
let's say that we can do it at least in
two ways so one way is
splitting the work into 15 pr's of 20
lines of code
and the other one is just giving one pr
of 300 lines of code
now in this kind of system the
cumulative lead time for these 15 pr's
of 20 lines of code is going to be way
longer than the cumulative lead time
that is going to take us to push these
300 lines of code through our system
now i'm not saying anything about the
amount of value that we're introducing
with this but if the system is choked in
a sense that you are not even able to
push the things soon out of the door
then you are not able to get the value
as well in the cases when you get value
from the changes that you introduce
right
so
what this means for me is that actually
on the lower
side of the spectrum of the size of the
pr we lose throughput right so the
smaller the pr
the more they get the weight
inside of a system per size
and
we have
the the behavior that means that we need
to trade off between two things right
one is the
uh quality that we talked about and the
other one is throughput
and when you think about it actually um
when the the interesting thing here is
when when we when you think about the
whole process is that as we reduce the
size of the pull request
the processing time tends to go down
linearly uh or or
stays constant processing time precise
why because we are mostly dependent on
ourselves when we code right but the
thing with the wait time here and
the review process is that
um the as we reduce the size of the pull
request the wait time precise goes
exponentially up as we saw it and why
because if you have smaller peers that
means that dependency starts kicking in
with other people sooner right and more
often so it becomes more expensive which
this then this ratio weight
to processing time goes exponentially up
because the wait time is going
exponentially up
which affects the flow efficiency cause
it to plummet and it also means that the
throughput is going down
so
from one side
we have uh
inability to build the quality in with
big prs
on the other side
we have a problems with the throughput
and uh
the smaller the pr the less
uh throughput we get from our system and
that's this eternal
uh battle between speed versus quality
throughput versus quality throughput
versus stability whatever you call it
um for anyone that has had a chance to
read
john reiner's book it was mentioned
today i think at least once
the principles of product development
flow and i really recommend it
to managers but also to engineers
this problem is actually about this um
optimal batch size
of
curve which is called u-curve of the
optimal batch size your curvier
and here we were talking actually about
the transaction cost now
it's a pretty washed out slide but the
the
the optimal batch size is here right so
you can see it from from this point of
here is
the batch size because it's trying to
optimize between the holding costs that
we see here
and the transaction code that we just
talked about
um
and this led me to
um think about the work that has been
done as part of the
accelerated book which is a summary of
dollar research state of devops where
you know this for me the biggest
conclusion was that instead of thinking
either throughput or stability uh
there's actually
you know um
both rupert and stability when one goes
up the other one also
goes up and vice versa so um
we tend to hear all lots of the times in
our english industry that there's always
trade-offs and i agree with that but
there are some cases where the
trade-offs don't actually exist because
the underlying assumption is flawed so
it's really important how do we frame
the problem in the first place
and
um i think we can have our cake and 82
at the same time so if we try to work
backwards from the problem that we have
so we're saying that with the cost of
code review precise is going
exponentially up then the throughput is
plummeting
what do we need to do in order for
throughput to be constant right
so
if we
need to
keep the throughput constant as we
decrease the size of the pull request
and again you want to have smaller pull
requests
that means
that the costs of code review per line
of code or precise
must not go up exponentially actually it
needs to stay constant right in order to
do this
um
the
actor's reaction time when i say actors
i mean authors and reviewers
in the whole process
needs to go down
as we decrease the size of the pull
request remember emma and luca they
weren't able to react immediately and
that's what is driving up the cost of
the code view precise
so as we decrease the size of the pull
request they need to react exponentially
faster and faster and faster in order to
not lose the throughput at the end and
in order to do that we need to increase
their availability
meaning
hearing theory little's law for anyone
that is familiar with it
less work in progress in the system in
order to keep
people less busy which means that they
are able to react sooner
so if we go back to this ticket number
one and i
we just pull
out
and we see anything
oh yes
um just focus on ticket number one and
this is a timeline of the events and we
let's say that we have this
size of the puller quests and let's say
that we want to decrease the size of the
pull request in order for us to not to
lose the throughput as we decrease the
size of the pull request m and look need
to react faster right
if you keep reducing at one point they
will have to react immediately
or not to lose the troop
and that was the
kind of conclusion of my study in order
to not exponentially lose the throughput
um
while reducing the size of the pull
request people need to get
exponentially closer and closer and
closer in time which at one point leads
us to continuous code review right
and
if we go back to this causal loop
diagram here
there was a problem
this conflict that we had
in this point right
which was caused by the number of
interruptions that we get to see in the
system
but you cannot get interrupted if you're
not doing anything else
so if you're working on the same item as
the other person or you're not doing
anything else that's not really an
interruption for you right
so
that is a segway this
parallel universe that they call
co-creation patterns i hope that all of
you at least have heard from peer and
more programming
but interesting thing here
with working together when it comes to
this whole process
is that
because the
we have someone sitting next to us
giving us
review immediately
the cost of code review per line of code
is actually minimal because they are
providing us the the wait time is zero
effectively right which means that now
we are still not able to see this there
is this optimal um
optimal batch size which was here and
then because we reduced the transaction
cost we were able to shift the optimal
size of the batch to the left all the
way to the left
um
and i was thinking okay how would this
look like in case we did
pair more programming in case we
co-created right so when we think about
the wait time precise if i'm able to get
the feedback immediately right that
means that the wait time per size is
actually zero
and effective size of the pull request
if your pro request is dependent on the
code view is one line of code or the
atomic change that the person that the
driver is doing is making
now when it comes to the engagement
the engagement tends to stay
the same or at least goes up why because
the um
the the feedback is more timely
and the communication is also variable
so the broadband of the medium is higher
plus we're also able to fast course
correct each other
and
i was thinking then you know okay that's
the case and we get actually the
continuous code review as a byproduct of
co-creation
and
thinking about all of these metrics here
i was thinking okay you know can we find
a number that we can use
to kind of describe um the things that
we're trying to optimize for
so we want the size to go down right the
pulley press size we want the wait time
precise also to go down
and we also want to engagement precise
uh if not to go up then at least not to
go down to stay constant test but it
used the size of the pull request right
so you might have noticed this
bar here on the side which is a
thing that they called pr score
and
if we try to map into a formula these
things that we're trying to optimize for
and their relationship
so you know there's a size uh there's a
wait time in seconds and the engagement
and for a co-creation when we get to the
feedback immediately the effective
size of the pull request can be to the
point that it's one line of code uh wait
time is zero and engagement is whatever
constant or it goes up i divide by one
plus engagement because engagement can
be sometimes
zero non-trivial comments and i don't
want to divide by zero
actually no one wants to divide by zero
and this whole thing evaluates to zero
now the thing was i was trying to
calculate this also for the async code
reviews and the numbers were so high
that they had to use the natural
logarithm here so
you to evaluate it
at the end and the
the pr score for the co-creation also
evaluates for that to zero because
natural logarithm of of one is
zero
um
so
that's why i
tend to say that the optimal size of a
pull request
is one line of code that is reviewed
immediately as is being typed
and i honestly don't know a better way
to achieve it than by pair and more
programming so
the the thing with it is that as a
byproduct we get this continuous
immediate code reviews
i'm not going to go into
psychological safety
trust vulnerability that tends to be
built with co-creation with working as a
team together that's a completely
different talk
but you get so many benefits out of it
so i was plotting this on a
on a
time series and here we can see a data
set of of a bunch of pr's
uh on the y-axis we can see a score and
it's again
on uh
um logarithmic scale and i think the
lowest one was maybe around six or so
right and that's one world that they get
to see
and then the other world world is a
continuous code reviews pairing or
moving which has
this result as a zero and if you try to
optimize for these metrics that i talked
about then
actually the best result that you can
get is zero
so
throughput or quality
uh
i would say both throughput and quality
because some assumptions don't hold true
and then i was also thinking you know
what if we try to define
peer programming uh with this another uh
this orthodox definition of people
spending twice as much time working the
same thing
but then the wait time is zero because
we get immediate review
then how would the world look like and
this was this was just one of the
examples
of
how sooner would we be able to finish
which is really important because of
that learning cadence that i was talking
about
and why is important in order to be able
to deliver
value to our customers sooner in order
to at the end hopefully outrun the
competition
so in this case it was 3.3 times the
numbers again are not really important
and the accuracies i'm sure not 100
but they are a really good kind of
conversation starters for teams to start
evaluating the way that they they work
and with that um i just wanted to say
that we've been at least that's my
impression that we've been told all
along that we'll achieve more if we
limit and
delay our interactions but
i hope that now you also have a data
informed reason
not a reason not to believe that
and with that i would like to thank you
for
your attention your time
yeah
um
feel free to rate the talk uh outside
that's also what i've been told by the
organizers yeah thank you for your time
thanks
you