thank you
okay
so it's time to start and then welcome
everyone my name is Tobias and my name
is Martin and we are from Omega Point
Gutenberg we do with Security reviews
and the penetration tests and we also
work as developers the writing and
maintaining code and we also help teams
with the application security
so our thinking here today is a result
of years of developing applications as
well as performing uh Security reviews
and then in the presentation we will
talk about how automated tests can help
us build secure apis and the code that
we will show is available at GitHub it
will be the demos will be in.net c-sharp
but the the concepts that we present are
valid for any framework and we also have
a repo in Yoda
so uh we rarely see teams writing tests
for security and we believe that's a
missed opportunity uh it's both quite
simple and very powerful to write tests
that verify that your defenses are
working correctly and we also strongly
believe that tests can help you better
understand your domain and its
boundaries and today we'll talk about
how to choose which test to write and
we'll also show you a bit of code of
what that could look like yeah first we
need to talk about why pen tests tend to
find the same vulnerabilities over and
over again
yeah so this is the WASP top 10 list for
apis
we know that it's from 2019 and it's
still very accurate it matches our
experiences I think quite well
API security issues and how to deal with
them has been known for several years
and there are of course many reasons for
that one could be
um not enough budget or team resources
to deal with Legacy code which might be
hard to address but we often find that
vulnerabilities originate from unclear
non-functional requirements and lack of
tests with security in mind so if these
issues was found during development or
testing we would simply call them bugs
not the vulnerabilities
I think it would be great if we could
catch security bugs just as early as any
other bugs before the application has
been deployed into production yeah it
showed so here we have a pen test report
where we have findings represented by
the red yellow and the green lines over
there and we address them or refine them
to bugs in in the backlog but um even if
we represent them as bugs that there is
an important difference between
functional bugs and those security bugs
functional box they will often be
discovered by users so if they end up in
production while security bugs they
won't be discovered by users and
attackers who exploits them they will
probably not report them as well and and
probably not be detected so the
vulnerabilities will stay open for a
long time
yeah so uh if we think about
vulnerabilities as normal bugs then
security is just another non-functional
requirement and should be part of your
quality assurance process with backlog
refinement
prioritizations and all the other things
that the teams do so in this case the TD
math tagged Security banks and we can
see that the team has prototyped the two
most critical findings for the next
release
political hand in hand
and how can we as the developers assert
high quality in the applications we
build
yeah
so our industry has some very well
established patterns and practices for
quality assurance and one of them is
test driven development and we'll try to
show how to apply tdd to application
security to help find those security
bugs during development and not off the
deployment to production yeah so note
that we are not saying test first but we
are saying tests before deployed to
production
and that sounds completely obvious and
simple but we do find that while most
teams do Implement automated tests these
days we rarely see test aimed at
security so well written test cases can
help us identify the boundaries of our
functionality and if we find that we
have a hard time writing good clean
tests for security then I think there's
a high probability that we have problems
with understanding the domain that we're
working in yeah however in another way
you can't secure what you don't
understand and the tests will help you
understand and Define exactly how a
function should work not just the happy
case and so now let us look at how we
can apply it Theory thinking to our
application security yeah
so Elizabeth a context
um we're thinking about for this
presentation we will focus on apis
obviously so picture A system that
consists of a set of apis and we have a
set of callers that is consumed in those
apis
some are ordinary uses some are high
privileged administrators
and some more services perhaps other
apis that are talking to us and we need
to restrict access to these apis in some
way so that each caller should get at
the least amount of access possible
yeah so if we look at basic requirements
for this context then we could express
this in the following way
um
callers should be able to consume the
apis
uh this would meet any functional
requirement and we would typically
verify this during testing and
development and when it works we'll test
the green we can deploy to production
right yeah well um we note that the
requirement here is not perfectly clear
it allows for the ordinary users to
access functionality intended for
administrators and services and instead
I think that we could express this
requirement as the caller should only be
able to consume the apis and
functionality needed and we know that
this sounds easy and completely obvious
but we often find that teams do not
express
negative requirements like this and it
is essential for security that we
identify boundaries and our requirements
yeah because this is what attackers do
they try to misuse the functionality
that we provide and break out of
boundaries and execution contacts but
it's hard for teams to switch to the
mindset of an attacker and try to misuse
their own system it's more natural for
developers to think in terms of
non-functional requirements and the
domain rules so if we do that we can
limit the tech vectors without being
experts on attacks and so how can we
verify this only yeah so I think that
for this uh we typically use negative
test cases uh the users should not be
able to consume API 3 that will assert
that
um we can only reach API one and two
um I think that's another good example
of these negative test cases is tenancy
isolation perhaps the user should only
be able to access data that belongs to
her organization
we rarely find requirements or test
cases for tenancy isolation but most
teams that we talk to and work with they
identify this threat among their top
concerns and it's also the number one on
the OAS top 10 list for apis
yep so
yeah so we mean with a secure API
and you can of course implement the
secure API in many ways but you need to
handle these six steps in one way or the
other for all requests yeah so this is
typically roughly what we are looking
for when we review code from a security
perspective and based on experience we
believe that you do need to address all
of these steps in order to get a strong
solution
yeah so in step one we validate that it
is traffic meant for us that is a valid
request valid HTTP and we should get
this from the web server and the
infrastructure protection step two we
need to validate the token for API that
often means that Json web token and this
which should do by using a
well-established library or component we
should not implement this on ourselves
so step one and two should only be
configuration and support from the web
framework we use not our own custom
application code and then for step three
we transform the access token to
permissions we need to some mechanism to
go from authentication method in the
token represented by claims to reach
fine-grained permission models and then
we're down to the domain logic and in
step four we will validate the data in
the request that is semantically correct
and in step 5 we verify that the caller
can perform the operation requested
yeah and we should note that this is a
model and the order between step four
and five might be different in different
implementations
we should do basic
authentication or authorization at the
routing level and this could be done in
a Gateway in front of us or a in the API
using a controller if you're using.net
and but we should also note that this
does not replace any
token validation in the API or
authorization in the domain layers
for a strong solution we need to build
with multiple defense layers
and the the core business domain layer
should not assume that other layers has
done security like input validation or
authorization
so in the final step five we verify that
the caller has access to the actual data
requested that's a very common mistake
to miss which lets us break out of
tenancy isolation yeah so how do we find
out what kind of tests we need yeah so
um for that a good place to start is of
course the owasp asvs but uh we also
like the WASP web security testing guide
and we think that these two resources
can be used as a sort of security
testing Baseline and that will help us
identify test categories and for these
for this demo we've listed the most
relevant ones for us
for your application you might identify
additional categories we know for
example that we do not have compliance
and regulations part of this list
performance might be higher for your
application
yep so let's look at how we can use this
and here we have the API and we have the
test categories and in the DML we will
add tests to verify that we cover these
categories we will use we will have
system tests that run against the
instance server API and we will have
unit tests where all Integrations are
mocked and we will start with the system
tests and
they will concern step one and two in
our API model and test categories
authentication basic parts of
authorization and Transport Security so
we will start by looking at the API
let's drop down to some time for demo
time for demo so I've got an API running
on my machine here uh it's uh
running on localhost I've got 5000 and
please note that it's 2023 I'm running
https only there's no need for HTTP
anymore I think even for local
development
so we can look at the controller yeah
let's look at the controller
so this is a standard controller in
csharp.net we notice that is that big
enough
works yeah great so the round bass ramp
for the controller is API slash products
I've got a an implementation of a single
endpoint uh the endpoint takes a string
from the user part of the route it
passes that ID
to a repository abstraction and if we
find database entity we map that to a
domain object and we return a 200 okay
with the product
so uh when we review code uh this can be
a lot more complex than this obviously
there can be a lot of things going on in
the controller but this is the essence
that we want to lift out and typically
what we're looking for as well it is
obviously a weak implementation there
are some pretty pretty glaring
vulnerabilities here for example
um simply returning the domain object
back to the caller when we are leaving a
trust boundary like this for an API we
are typically looking for dedicated data
contracts to represent the data
we can also see that the input string
here the user input is passed directly
to the repository layer that might be
fine the repository implementation could
correctly encode this data use
parameterize queries for SQL statements
Etc but at this point we simply do not
know that an injection is common so yeah
yeah
I know where my money is for that
and also perhaps the most obvious one
there's a complete lack of Access
Control here
perhaps there's 99 other endpoints that
are perfectly implemented with full
access control and with just something
gone wrong for this endpoint and
is simply missing yep and before we move
on we would really want to point out
that injection is mitigated with correct
output encoding for the given context
and the input validation will only limit
attack vectors but we should use both
when we build strong safe implementation
of apis
and also we could also note that there
were a lot of Overstock turn things here
one is excessive data exposure returning
too much we had injection and we had we
have different types of broken
authorization here as well
and um yeah so we can look at some
system tests this yeah so we have a set
of system tests that we wrote for this
purpose we should start yeah
let's see how it goes yeah
and um
yeah well notes no it's not terrible no
some some red some green we can start
looking at the health tests yeah so
let's have a look at the health tests so
a very important part of any API is
availability
what we've done here is that in addition
to the product standpoint we have two
endpoints for availability liveness and
readiness and the first System test says
that when I have an anonymous HTTP
request and request the liveness
endpoint I should get a 204 no content
back
if I do have an anonymous client and try
the red in endpoint I expect a 401 and
authorized back and
if I am authenticated I expect the
Readiness endpoint to return a 200 okay
the version number of the software that
I'm running and some information about
the.net runtime that my software is
running on top of and here since we are
authenticated in this Readiness endpoint
we can do deeper system checks like
accessing a database or things like that
so we often see the example for
Readiness endpoint you check that the
database is available perhaps you query
Downstream Integrations that you use and
we often see quite a lot of information
being passed back to the caller as a
result of that and there is absolutely
no need to expose that to an anonymous
an anonymous client no because we should
because that is quite a common mistake
as well that you have an anonymous live
that starts out simple as a pinion point
and then it grows and returns more and
more and maybe you could use this to do
a denial of service of the system yep
then there's a system test that when we
have an invalid token we expect that the
endpoint should return a 401
unauthorized
uh good examples of invalid tokens is
perhaps a previously valid access token
but that one that has expired another
good example is an ID token that one we
see quite often I would say ID tokens
should never be useful for API calls the
API should only accept access tokens and
the another test case would be invalid
the signatures like the non issue that
is quite common depending on the
framework you're working with
and then there's a set of assistant
testers I'm not running uh while I'm
working localhost because they require a
reverse proxy in front of them
the first one is that I'm not accepting
traffic to http
the next one is that the TLs version
that I'm requiring is version 1.3
and the final one is that all endpoints
should return a small set of security
headers or wasp guidelines recommends
that we do return a set of security
headers even if you do write an API they
are not that many and they are quite
easy to implement so um
yeah that's a good good recommendation I
think and yeah with that we can look at
the error tests yeah right so we have a
system test that uh when an endpoint
throws an exception we expect a 500 and
an empty result with no additional
information
and for a high quality API it can be
quite tricky to make it throw an
exception so what we typically do is
that we add an error controller whose
sole purpose is to throw an exception
and that makes it possible to write a
simple system test like that please
remember to require authentication yeah
and verify that you don't leak any data
because that's quite common as well that
you when error occurs you want to
tell the developer what went wrong and
then you might give them connection
strings and stack traces and a lot of
information so be careful the correct
way to to figure out why we throw
exception is probably a look at logs and
not return data back to the user no
uh so we can look at the products such
as the system for the production so uh
we have three test uh simple system
tests for the products endpoint uh when
I am authenticated and have the correct
scope and query for the product I expect
a 200 okay back
when Anonymous and querying for the same
product I expect a 401 on authorized
pack and when correctly authenticated
but with the wrong scope I expect a
rescope here I expect a 403 Forbidden
back so we can note here that the happy
case works but not the negative case
test cases and this shows the importance
of the negative test cases because
without them we might have deployed this
to production with an open endpoint
developers rarely test their
um or at least manually test their apis
without authentication because
yeah
thanks
yep so we can go back to the slides and
the test that we have shown them is
um as a covered data categories
authentication basic parts of
authorization and Transport Security of
course are you that validating a Json
web token isn't authentication it's part
of authorization but for this demo it
really doesn't matter to distinguish
between the toes so we can yeah let's
move on and look at the unit tests yep
and the first set of unit tests are for
the controller and for the
responsibilities that the controller has
which is to do a basic input validation
and handle the result and return the the
data and also the correct response codes
so we connect this to the test
categories the data confidentiality and
the integrity and now we will have a
look at the test for the product
controller yeah
so let's close the system test and look
at the unit test for the products
controller maybe start by running them
again yeah yeah that's good
[Music]
but then we know that we have a weak
implementation yeah so uh let's look at
what we have chosen to write okay we can
note that they have the case works yeah
it always works it works it's it's
happening yeah so the uh the first test
is that when we set up the controller
for success I expect saying okay object
result back from the controller uh for
data not found I expect the 404 file
results and when I set up the controller
for no access to operation I expect the
forbid result back uh maps to 403 and
when I have no access to the data
we can typically choose between two
valid HTTP status codes uh 403 Forbidden
which is correct so I mean there's data
there but I'm not allowed to use it or
the 404 uh if I don't want to reveal the
fact that there's data there and
for this demo we've selected the not
found 404 result
and then there's a test that verifies
that when I try a few uh samples of
invalid product IDs I would expect a 400
pound request object result pack and
then finally
um when I do get a result back I expect
to get a dedicated data contract back
and not the full domain object and most
of that fails so yep and we will address
all these red tests later when we go
back to the slides and look at the next
categories of tests and they will
concern step 3 where we transform from
the Json web token to permissions and
they will verify the permission model
that we have for this DMO API which is
that you should only have access to
products for a given Market
so let's start by looking at the
permission service yeah so if we look at
the
permission service so
um yeah so so
um many Frameworks and net in particular
kinds of leads us down a path where we
have a principal object a user object
injected into the request Pipeline and
that user has a set of claims in it
represented by dictionary string value
key perfect and that's the object that
we can query when we to do our access
control on top of
it doesn't have to be like that uh
in this code we've instead selected to
declare a permission interface
that contains at the top for properties
that reveal the identity that we pick up
from the incoming access token regarding
the user and then here first a set of
simple Boolean properties if I can read
products and a simple method that with
figures out if I can read this
particular product and we think that
makes for a much better developer
experience it's strongly typed you will
get intellisense in your IDE and don't
have to work with these key value
strings
and also lends itself perfectly for
writing unit tests yeah and we should
note that this often involves a service
or database lookup where we use those
identity properties from the token to
look up the permissions at and maybe
make authorization decisions like you
can read products
and this gives us a centralized pattern
for for the permissions and and the
access control and it can of course be
externalized using a service that we
share between apis or things like open
policy Aliant and so on but but we use
an in-memory implementation in in our
DMO and and that's often fine as well so
it could be yeah so should we run the
tests yeah so let's look at those tests
and start by running them
see what we've got
yeah all green strong implementation for
once so let's have a look at the test
that we selected for the permission
service
it is that the unauthenticated user
represented here by the claims that we
would expect to find in the token should
properly populate the identity
properties of the permission service
the second one is the unauthenticated
users with the correct scope should be
able to read products
uh next test is an authenticated user
should be able to read products from my
market and with that done we can move to
the negative test cases the cases that
identify and declare the boundaries of
my functionality the first and the
obvious one perhaps is that if I have an
anonymous user that do not have any
claims in the token so this is then I
expect a an empty identity and I expect
not to be able to read products
and uh
the next one is that an authenticated
user but with the wrong scope
um I would expect not to be able to
reproduct
uh
the next one is that an authenticated
user correctly authenticated should not
be able to read products that's not part
of my market and I think that those
kinds of tests are hugely valuable and
nicely documents and uh
summarize what we would expect from the
information model in particular this
last one this is sort of multi-tenancy
and either and things like that that's a
very very common problem in apis and
this kind of pattern of declaring a
central permission service or policy
service like that enables us to write
these kinds of tests in one place it can
be quite tricky otherwise to find this
uh this documentation across your code
yeah so all is green and we can go back
to the slides and the final test
categories
and the final set of tests is for the
product the domain or service layer
that's step four or five and six in our
API model and now we will address the
remaining red tests that we have seen
and so that we can assert that we have
well-defined boundaries and cannot
misuse the API and we will cover all
test categories except authentication
and Transport Security
and we will now start fixing the red
tests and weaknesses in our controller
so we can go back to the controller yeah
right and we will start with limit
attack vectors for in action by
introducing input validation so let's uh
let's try a naive approach here and
simply start with the fact that I'm not
accepting uh noodle data
no I will not accept
an ID that's too long
and I will not
accept any input data that is not a
letter or did you like that
so uh if that happens then I will return
I thought no I will not redirect
a bad request yeah thank you that's
funny valid uh ID yep like that I think
yeah looks good uh let's try it out yeah
and run and run the controller test
again restart the API
let's run the controller I want a lot of
Red Pass there and if you remember so
now let's see if we improved the code
anything yeah the input validation tests
are green so actually this simple code
it captures the requirement we have for
the products API but um this kind of
code is it's kind of if the if there are
a lot of endpoints it's easy to do
mistakes with this kind of code so we
want to use a stronger pattern and to
limit the risk for that and we can use a
pattern for from the main river design
or domain driven security where we
introduce domain Primitives
so because our product ID isn't a string
that can be can contain any character
and it should always be treated like a
product ID so so we have a domain
perimeter for a product ID implemented
it's not big
um
as you can see uh I got the same
validation here that I just implemented
in the controller and the Constructor
here will assert that that input data
conforms to that semantic validation and
if it does not it will throw an
exception so it's actually not possible
for me to create an instance of this
domain primitive without also conforming
to our rules for what the product ID
would look like uh and we have some unit
tests
very easy to write tests around our
class like this such domain Primitives
Primitives lend themselves very nicely
to unit testing uh of course we've got
the happy case uh two valid IDs here
that should uh the Constructor should
accept and should work properly
and then we have again the negative test
cases verifying the boundaries of our
functionality
an obvious invalid ID that contains
spaces
but then I want to try some some some
injection strings so what we often do is
that we resort uh to
um
um a resource that's a lot of resources
on internet for injection strings we
like the GitHub repository big list of
naughty strings
they maintain a list of injection
strings so that we don't have to figure
those out ourselves and I've just
selected a few of them that I thought
was relevant that repository contains a
lot of a lot more information for
example some tricky names that tends to
crash systems uh that's that's quite fun
as well
yeah so what we do here is that we try
all of these and see if the and we
expect in our test that the Constructor
would throw a an exception yep so we can
apply this and we can go back to the
controller yeah
so instead of this code then I can
simply use my domain primitive
so next ID is valid
like that and I would then typically
quickly move to the products ID doing
primitive
in my code
okay
like I think yeah
yeah so uh so that's that's good yeah
and we should note that using the main
Primitives is more than just input
validation but we will not cover stage
chaining functions and domain Logic for
that today
and that the same reasoning applies to
other domain Primitives like an address
or a product name or or something like
that
so now let's look at authorization and
we could of course add code for
fine-grained authorization in the
controller
um like we did with the input validation
but
um this does not scale well when the API
grows in features and we have more and
more endpoints and and we don't want
that kind of logic here in the
application layer so we will introduce a
product service yep so um because of my
product service here it's a simple
interface uh it's a get my ID method as
you can see we're only working with
domain office now so no strings only
domain objects objects from our domain
we will return a product and an
indication of the results so let's have
a look at the implementation and now you
we can use the permission service that I
showed earlier so instead of working
with string claims from the user
principle we can instead work with the
permission service
so if this permission if this calling
user cannot read products then we would
return a null product and a no access to
operation indication
and if we do find the product in the
repository we will check if this user
can read this particular product and if
she cannot we will indicate that this is
no access to data and otherwise we are
all good we return a success and the
actual domain object yep and if we go up
again yeah to the beginning we want to
note that the order is is important
because we we want to abort invalid
requests early and without consuming
system resources or doing complex domain
logic so we should always do input
validation which we can see from the
product ID and access to the operation
as early as possible before any domain
logic so if we do not have access to the
operation we will not even call the
report make the call to the database no
because verifying access to access
access to data might be really complex
you might need to work out the
relationship between objects and so on
maybe like a search function so that
will be the last thing you can do here
before returning the object
and also note that we do we do logging
for monitoring and accountability so
that is also important to verify in in
unit tests that you log when you shoot
yeah and we note that we only log with
the domain objects as well
yeah and also that we note that we that
we read the data because yeah for for
some systems accessing data can be the
most sensitive operation there is
yeah so uh let's go back to the
controller and use this service yeah
right so let's get uh oh sorry let's get
rid of the
[Music]
um
vulnerable code and using our new
service we can declare the product
and data result
and let's call our service
get my ID
and D like that
yeah and I think that with that in mind
we can now return switch on the data
results
and let's start with the easiest
staircase that is that we've got a
success
[Music]
um
in that case I would like to return an
okay result with the product
the the next obvious uh result is the
not found part I would return a 404 not
found
and then we had the
um no access to the operation typically
a 403 forbid
and then we had the no access to data
and we selected the 404 I think yeah
that's true correct yeah
oh
semicolon thank you and in all other
cases I will simply throw my hands in
the air give up and throw an exception
and my exceptional choice is an
environment operation exception
yeah because this really is an exception
you should never get here
so um looking good I think I think so
it's um it looks good and we can try and
run the controller test again and see if
we have improved anything with all this
restart the API let's hope so
yeah and uh yeah almost pretty good uh
we've got one failing test and that's
the test that we decide that we are
using a dedicated data contract which I
am not I'm still returning the full
domain object
so let's uh change that and use the
excellent
Auto mapper to
map with that domain object to a product
data contracts like that
so uh let's have a look at the data
contract itself to see what that would
look like so this is typically something
uh what we're looking for in our case uh
the data contract only consists of two
elements I re intend to return the ID
and the name of the product of course
the domain now we can contain a lot more
information and a lot of sensitive
information that should not never leave
our trust boundary yeah so this is the
pattern for mitigating the excessive
data exposure vulnerability so let's
restart that API let's have a go again
see if we're good
yeah
yeah so um always going there so let's
run
um the system testing and we can run all
the unit tests first yeah we could we
could we could but we are what the
states yeah they're all green if you
noticed uh all of those yeah it's very
hard to notice that and now we can run
the system test again but we have some
realm from before
and see how that goes
and um no almost almost we have the
three left there and if we look at the
two in the health controller they are
red because the authorization we added
it was used for the product service and
the product controllers and
unfortunately.net does not have a secure
default that's part of the templates to
have a strong solution we need a
mandatory pattern a deny by default and
forcing authentication and basic
authorization maybe for all requests in
other Frameworks like um your Spring
Security we get this by default but
in.net we need to add some code yeah
right so this is the program that hosts
our API this is what it looks like in S7
and as you can see I've added
authorization but as you said the
default is unfortunately to allow
anonymous calls we need to change that
default and the default is there to
ensure that the HEPA case will work from
the beginning
so this is what it looks like I set a
default policy as well as the fallback
policy to require an authenticated user
and use jvt better defaults and
so now we will require the better schema
here for authentication yeah for all
incoming requests so that's so let's try
again try those system tests again yeah
and uh
now the HEPA case doesn't work anymore
so the liveness that way now we need to
open up that endpoint because so now we
can decorate that we don't allow
Anonymous attributes and we have this
default by deny yeah so right now so
this now requires authentication which
was more wanted for the liveness I need
to decorate that with uh allow anonymous
and now perhaps finally
oh yeah let's run again
right yeah so we're all green and almost
done but for a strong implementation
according to best practices and over
spare asvs we should also apply
authorization of the routing level not
just at the resource level and we can do
this switch support from the web
framework in the controller or we could
do this in a Gateway in front of our API
or both with.net we will use the
policies and attributes with the Java
springing security we would use
pre-authorized tags so let's add those
policies so I just added two policies uh
and this is the name of the policy and
what it does is that the it requires
that the scope products read is present
in the token so if I go to the product
controller I can now decorate this
endpoint with an authorized attribute
and say that I want that policy in
particular oh not with double yep so
this gives us a way to abort the request
really early before we enter the
controller so the net asp.net pipeline
will now if the incoming access token
does not contain a price rescope I will
not even get into this code so
and we add this for it depends in depth
it doesn't replace any authorization in
the domain or service layer
and uh yeah this is it yeah pretty good
yeah I like it yeah it's uh we could of
course use more features in the web
framework or the language to get maybe
fewer lines of code and and so on but
this is sort of a clean implementation
that demonstrates the important security
um Concepts that we want to show here
for an API
and yeah we could not a few more things
that we use explicit routing we assert
this only accepted the get request and
that we get the data from the rotor so
we don't have any Mass assignment issues
yeah and um yeah so all is green and we
can go back to the slides
and um yeah in this demo we have showed
that with a reasonable effort we can
cover basic parts of security testing
and apply tdd to application security
and that we meet those security
requirements and address most of the
issues on the ovas the top 10 list yeah
and we think that none of the tests that
we have shown here are particularly hard
to write it might be among the easiest
that you have in your system but they
are selected with a little bit of care
and thought to verify the boundaries of
our domain
yeah so let's uh summarize yeah
um first a note on complexity uh when we
do security reviews uh we usually find
more vulnerabilities
than in more complex systems compared to
simple systems and we feel that's not
just because of the size of the system
where more functionality will give us a
larger attack surface but also because
it's difficult for a team to keep a
system secure over time that they do not
fully understand
many of the vulnerabilities that we find
originates from
unclear non-functional requirements and
missing test cases allowing an attacker
to break out the boundaries and abuse
the application so you can't secure what
you don't understand
so we believe I strongly believe that
tests will help you understand and
identify these boundaries and we think
that automated tests will make sure that
your rules and defenses are applied and
implemented correctly and we feel that
this is absolutely vital for security
yeah and the things that we have shown
here isn't that hard to do security is
hard but the things we have showed isn't
that hard to do and uh so if all teams
wrote those kind of tests then we as Pam
testers will have a much much harder job
and the bar for the attacker has risen
significantly so uh with this
presentation we hope to have inspired
some of you to perhaps do more tests
with focus on security
maybe start with a few system tests that
tries the endpoints with an anonymous
call uh maybe even run those tests
against the production environment
um yeah no so uh but um will this really
scale wasn't that a lot of tests for
such a small API yeah it's just one
endpoint
one day yeah there was a lot of tests
but but it always our investment in
security should be proportional to our
business and based on our experience the
things we have shown here with all the
tests are a reasonable level of security
for many businesses really no apis
should have vulnerabilities from the
ovas top 10 list no
um so
these and many other important patterns
are described in this book written by
our colleagues Dawn and verdonians and
uh you might have seen them present here
earlier today
and it's a great book which we recommend
and if you don't feel like reading the
book then island of the doll hella great
Back to Basics presentation on NBC
security last year demonstrating some of
these Concepts which we also recommend
watching on YouTube
so uh if you want to have a closer look
at the code that we presented here today
these are the GitHub repository please
take a look and uh
have a play around and I think that was
all from us thanks yeah thank you
if you have any questions feel free to
speak up
yeah
and what's your staff um let's be in
production so what's what do we feel
about testing in production uh we like
it
you need to be a little bit careful uh
what you do to your production
environment uh and any test that doesn't
require authentication is a great uh
candidate for running against production
as soon as you require authentication
you need to think about what you're
doing uh that needs means that there is
authentication for production
environment somewhere in your cdci
pipeline in some way you can do that but
you need to pay a little bit attention
to the details so the liveness endpoints
we will typically like to do to sports
production because it's there in the
production environment that's
availability is important the test
environment isn't that useful in that
way and we should also test that we
don't have Anonymous access by by
passing invalid tokens and so on we also
can do that but as soon as we succeed in
authenticating that then we need to be
careful so so System test against
production is hugely valuable of course
so man and if you want to reach the
radian point for instance or the Run
point then you should use a dedicated
test client with very limited
permissions to just do exactly those
things so but but you so you need to be
a little bit careful but yeah you need
testing production because
foreign
yeah so the question is uh to to instead
annotate the classes right uh for fluent
annotations instead uh certainly well we
had a little coding session here last
night doing exactly that uh this code
here today was written for clarity and
for demo purposes uh yeah but there is
also one one point here that if you
input validation could be simple like
the things that we showed here but it's
for some things that gets more complex
and that becomes the main logic and you
don't want domain logic in your
application layer and sort of mixed up
with the web framework so so you want to
keep the domain logic in the domain so
so even if you use those annotation you
can do that for basic things you should
have complete input validation in your
domain so and it's okay to validate
things twice like we did with
authorization it's the same yeah
great thank you
yeah just to think about the performance
but that's often not a problem yeah
okay great so thank you for listening
and have a safe journey home