hi everybody welcome back it's finally
time to finish that multi-threaded web
server we've been working on this is the
third video on this topic if you haven't
seen the first two I suggest that you
watch them first I'll link to bolt in
the description but of course I leave
that up to you so last time we added a
thread pool to our server to make sure
that if we get tons and tons of requests
coming in that that doesn't result in
millions and millions of threads being
produced which would cause our servers
performance to grind to a halt and our
thread pool really is just a bunch of
threads that are sitting there watching
a work queue that queue is where we put
stuff that comes in so as a connection
comes in we stick it on the work queue
and then when there's an available
thread that thread will then grab that
connection or whatever it doesn't have
to be a connection but in our case it
was a connection it can be any piece of
work that you want to thread to work on
it can grab it off the queue and then it
can go work on it so where we left off
last time each thread was constantly
checking the queue to see if there was
new work to be done and that's the
problem that we want to sort out today
because having your threads check all
the time isn't really great because it
means that our server uses up all the
available CPU cycles that machine has
even when there are no connections
coming in and that's definitely not
something that we want so today I want
to talk about how to fix this now one
obvious option that at least one of you
suggested in the comments of the last
video was to simply add a sleep
basically any time we try to get work
from the queue and we don't find any
then just sleep and for illustration
sake I will sleep for a second you could
of course sleep for less too but this
will illustrate the point clearly and we
compile our server and we run it and now
you notice when I send it connections
these connections are taking a little
bit slower than we saw before they're
taking somewhere between what they were
taking before and a second and the
reason is is that often when these
connections come in they have to wait
for one of these threads to finally wake
up to actually get the job done this of
course is because when the newer comes
in all of my threads are sleeping and so
we have to wait for up to a second for
one of our threads to wake up and handle
the connection sometimes it's going to
be fast if a thread just happened to
wake up sometimes it's gonna be rather
slow and yes if you sleep for less time
your server will be more responsive but
the problem here is that the more
responsive you make it the more CPU
cycles are going to be consumed and so
there's this trade-off between
how much CPU overhead you want and how
responsive your server you want and I
just don't find adding sleeps to be a
particularly satisfying solution to this
problem so I'm gonna remove that sleep
now instead I'm going to use a condition
variable now just to be clear I have
never been crazy about the name
condition variable because students
always get thinking that it has
something to do with an if statement or
some kind of logical condition and it
doesn't all a condition variable does is
allows threads to wait for some event to
occur a condition so when those threads
aren't needed they can just be suspended
and then they're going to jump into
action when the time comes for them to
actually do something which it sounds
familiar because that's exactly what we
need to happen in our web server
specifically you can do two things with
a condition variable a thread can wait
on it that means that the thread will
wait not doing anything until another
thread calls signal that's the second
thing that you can do with the condition
variable you can signal it the waiting
thread is going to wait until another
thread calls signal on that condition
variable okay so let's add this to our
little server before any thread tries to
take work off the queue let's just call
pthread cond wait that's the pthreads
version of wait waiting on a condition
variable which will make the thread wait
and tell it signaled then up where we
add work to the queue up here where
we're calling enqueue we're just going
to call signal and this is going to let
one of the threads that's currently
waiting no longer be waiting it's going
to stop waiting and it'll be able to
jump back in now those of you that
haven't seen condition variables before
might be wondering why I pass in the
mutex into wait and the reason is the
condition variables are designed to work
closely with mutex locks when the thread
suspended it also releases the lock and
if I didn't know other threads could
grab the lock to put work on the queue
and would be stuck so one thread would
have the lock and then nobody else would
actually be able to access the queue
because they'd be trying to grab the
lock and they wouldn't be able to
so when we call wait when a thread calls
wait it waits but it also releases the
lock and that allows other threads to be
able to access that critical section it
allows other threads to be able to put
work on the queue and then when it's
time to stop waiting
they'll reacquire the lock before weight
returns I hope that's not too confusing
but that's the way condition variables
work okay so let's see how this changes
things
so we compile our server and we run it
and now if I run top like I did before
you'll notice that the sir
is no longer consuming all the CPU
cycles in fact it doesn't even show up
here on the top near the top of the list
because all of its threads are suspended
just waiting for some work to come in so
these suspended threads they're not
using any CPU and now when a connection
comes in its once again handled very
quickly which is really nice but we're
not quite there yet because our server
still has an issue and to see it I need
to send a lot of connections so I'm
going to change the script the client
script that I've been using now we're
going to send 150 connections all at
once
and when I run it then you notice that
it starts off ok but then it gets stuck
so what's going on here it's pretty
simple our server is just too simplistic
so our threads are waiting each time
through the loop and our main thread is
signaling each time a new connection
comes in but when a lot of connections
comes in we learn something about
condition variables and that is that
they are stateless what that means is
that when I call wait my thread Waits
it doesn't care how many times signal
was called before it it waits now when a
lot of connections arrive and are added
to the queue signal is called but let's
say that the threads aren't keeping up
at some point there is no new work added
to the queue so signal isn't getting
called anymore
but there's a bunch of work still on the
queue and my worker threads finish their
tasks and they call wait and they wait
even though there is a ton of work to be
done
they won't grab any more work until
another connection arrives and thread
call signal this is a problem but it's
easily sorted out so to fix this I'm
just going to first call DQ and if it
returns null then that I will call wait
on my condition variable and now the
threads will only wait if they can't get
work from the queue that's the whole
point so now we no longer get into this
trouble so now I can compile it and run
it and now we get through all of our
connections and we have a working
multi-threaded server that uses a thread
pool it doesn't allow us to pile up a
million threads and it doesn't use all
of our CPU cycles while waiting for new
connections to come in and this last
part we got by using condition variables
now this is where I'm going to end my
multi-threaded server example but I just
want to be clear this server is not
without its issues it still has some
ways that we could improve it and we'll
work on that in future videos one of the
server's issues are that it's still
vulnerable to a particular kind of
denial of service attack so let's say
that a client connects to the server and
this client takes a long time to send
its message the server is not working
very hard because data is coming in very
slowly but one of the threads of my
thread pool
is going to be waiting and waiting and
waiting and just trying to get the data
off this connection now let's say that I
create a bunch of these really slow
connections they are now going to occupy
all of the threads in my thread pool and
my server now is occupied with these
really slow long connections and it
can't handle anybody else and so if I
decide to be really nasty I could just
bring this server to its knees by
sending it a bunch of really long slow
connections and there are some ways that
I could handle this specifically using
event-driven models and asynchronous i/o
and I am going to get to those in future
videos but not today because that's all
the time I have for today thanks for
watching subscribe if you want to make
sure you don't miss any future videos
and until the next video I'll see you
later
[Music]