hello can you hear me yeah I think you
can let me set the alarm for myself
because I always
overtime um hello everyone thank you for
joining me today um I'm so glad to see
you here and thank you for showing
interest in large language modos or
small language model or whatever you
call it foundationals
modos um let me introduce myself because
actually before I introduce myself
forget to say this do you remember the
octopus um for the World Cup prediction
back in
2010 Paul the octopus when I was doing
the slides in a way I thought about Paul
and he predicted um eight World Cups
game correctly and I feel that large
language model in way it's like Paul but
there's way more billions of
parameters um so anyway let me introduce
myself because this is surely my first
time meeting with you um all new
faces I'm Gia hello uh I am all the
places I've ever been I'm all the people
I've ever loved and who my exchange
knowledge with um all the books I've
ever read all the Tik Tok videos I ever
created all the communities I ever
volunteers and all the sport races I've
ever struggled to finish all the jobs
where someone was Taking a Chance on Me
and all of that plus the potential of
who I will ever be I am Gia now I hope
that you don't think that I use AI to
generate that because I read that myself
it's a little bit like you know but um
anyways I read that myself I know it's
been three days of um a lot of
informations love new framework if you
want to get a bit of a laugh right after
this um session I highly suggest that
you go to Diamond Cutter 88 on Tik Tok I
have viral videos that is really funny
and it will instantly make you laugh
anyway so that's me looking after your
mental health now do you remember your
first prom that you
wrote who remember no oh my God come on
don't be shy nobody
remember you know this in in order for
this SE session to be successful we
would do a lot of collaborative thing
I'm going to at the end towards the end
you're going to help me improve the
prompt so you will need to shout at me
your answer okay okay don't be shy so
okay that's fine some of you kind of
remember and some of you might
not um if you do remember what did you
do with the generated output of your
first
prompt you remember that okay yeah okay
keep that in your mind this was my first
prompt okay at the time we kind of went
through a little bit of um family drama
and I was about to move in with my
partner so I had to to message my Asian
mom obviously my mom is Asian and I
trying to ask chat GPT for my life
advice about hey what should I do how
should I frame the message and this
sounds really great right said 1 to7 I
even have to reassure her that our
relationship will never change I
actually put in my draft message that
she can come over anytime but actually
it was so offensive to her because she
was like well if your partner I'm not
going to go and hang out and you know um
play games with you it's not that like
like that in our culture okay and I get
that but that was me being so naive when
I first used large language models okay
that was about like a year orish ago
when it first came
out okay now I think because we kind of
talk a lot about the lar language models
in this conference there are many
language models actually who was at the
um gotot hansome and session this
morning that he show you there's
different um models um Beyond GPT
there's hundreds of models and it's
usually categorized by whether it closed
or open right um or how many parameters
um it it used to train the models and
you can see GPT um GIS and all those
things all like in there um it's
actually a really good website that you
can go into and have a look um and it
tells you all the details of different
models now what you're going to learn in
this session uh techniques tactics uh
patterns and all those things can be
highly generalized across many of these
different models I did try and it did
improve um the response that I get from
the language models so what's out of
scope today because otherwise you have
to sit here for um the whole uh the
whole day is the text to image model
we're not going to touch on it like
Dolly stable dfusion mid journey and
alike as well as multimodality I'm not
going to do the prompt for that um
because it's take times and also um the
behaviors are quite different in those
different models as well so maybe that's
something I can do in my next talks now
because you are an expert in latch
language models for all these three days
everybody keep talking about how it
works so this is my test to you all
right what do you think is one word
after this sentence
the best thing oops sorry oops nope you
cannot see it can you see sorry the what
do you think about the one word that
come after this sentence the best thing
about AI is its ability
to one word to code okay anybody
else to to automate to automate okay to
automate to create predict
suggest predict predict suggest okay
that's good good okay it's like some
sort of creation you know I asked my
student in a boot camp one of them say
take
over okay and I was like wow that's
going to be really bright for your
future but anyways they all they all
laugh but okay so it's actually say
learn okay so at the simplest level GPT
or the other large language models what
they're trying to do is trying to
produce reasonable continuation of
whatever text that it got so far
and when I say reasonable it means you
know when someone when you expect
someone that has read or see you know a
lot of people have written in the um
billions of web pages those sort of
reasonable so GPT look at all these
things with the um with the sense of we
have to match in meanings and that's
where the tokenization the embeddings um
trying to find the relations um in the
multi-dimensional space comes in right
and when you get um the output and some
of you that have been in um the keynote
for today you see how you can turn that
on in open AI uh playground so you go on
uh go on full spectrum and then it will
come up with the probability of the five
thing that it picks now in this case
it's learn so learn is um the likelihood
that it kind of see it over and over
again from the bajillions of web pages
okay let's do another
example 5 NS plus 5 NS equal
two it's like a 10 ANS 10 ANS yes 10
ANS it is 10 right 64 64% but you know
how recent breakthrough of you know gp4
and Beyond like all the models that are
really capable understanding the context
nuances this model can understand the
nuances in our language the joke the
punt right so actually
I was using GPT 4 I'll let you
read I asked 5 + 5 n equal to okay 10
ANS I was like is it not 10 NS I know I
love that jokes and you guys are so hard
to make you laugh why but anyway so I
was like well is it is it not 10 ANS and
obviously like if you look at the
response back if you use GPT 3.5 it will
relentlessly argue with you right that
is 10 but now it say hey it's not 10ns
10ns are the people that you know rent
an apartment from somebody else um it
looks like you know mathematic uh it
looks like a play on a word um but
mathematically the correct answer is 10
NS so I think that is um really really
fascinating to see now okay that's a
quick overview on the lat language model
because I'm scared that you will forget
what it means right um so let's talk
about trust before we kind of go into to
the techniques now you know every time
that when the new technology is
introduced there always this gap between
you and the something unknown you know
that that eky that awkward feeling when
do you remember when you like go into um
some stranger car on your first Uber
ride I don't know how many years ago now
but you know that weird feeling or like
when you stay at somebody else um home
for your holiday for the first time like
how weird is that or like when you put
yourself in a self-driving car for the
first time or recently when you copy
paste the output from GPT to your
performance review with your boss who
did that I did so you know like that
that kind of eeky feeling that we have
that what we refer as a trust Gap right
and we try it a little bit by a little
bit and we are able to cross that Gap
because each time it kind of increase
the trust that we have in the output so
we crossed this Gap and of course I
didn't invent all of this you know
Theory there's a person um actually
she's from uh Sydney uh Rachel Sportsman
she wrote a a book about trust in
collaborative consumptions about 10
years ago I'm not sure if anybody um
have read it but she gave a TED Talk in
Sydney and she refer um the trust model
as the trust uh trust is a bridge
between the known and the unknown and
when you take a Le when you trying to
use it and you kind of uh confined in it
you taking a trust lead in that
technology whatever that is doesn't have
to be large language model it could be
something else before um basically this
signify the tendency to take meaningful
risk while believing in a high chance of
positive outcome that it's going to do
the good thing that I hope that it's
going to do right so this shows like why
just enough trust is crucial for
Innovations um or adoption of any new
technology for individuals or an
organizations but how can we actually
facilitate trust to happen well it's by
minimizing risk and reducing the
uncertainty when we use the
technology so let's have a look what
does it take for us to test uh to place
trust in someone so this guy Charles
green don't know if anybody knows him in
the year 2000 he come up with this
famous trust models and there's like
variables into this there's credibility
plus reliability and intimacy of a
person right and depends on how much you
trust them or how trustworthy that
person is if he is more or she is more
self-oriented like only care about his
own or her goal then they become less
trustworthy even though super credible
super reliable super
intimate now trust when I look at this
and I was thinking that actually trust
is quite different different when it
come to you know artificial intelligence
that can generate human humanlike
outputs
so I'm casually proposing okay I didn't
do any extensive research on this but
I'm casually proposing this trust model
for generative AI like how trustworthy
it is it depends on you know accuracy I
want to know and this is based on my
experience I want to know if the output
provide accurate um correct upto-date
evidence-based information I want to
know how reliable the output is how
consistent how safe how private how
Dependable it is over time across VAR
tasks and condition that as it to um is
the output tailored to individual user
if I ask about marketing for Gen Z I
don't want the outputs to be for Gen Y I
know you think that they're the same
generation but actually they're not um
or transparency I want to uh I want to
know that when it gives me the
recommendation about my career
advancement
right what was the process what's the
context what the data that it had gone
into process I want to know like can can
can it tell me that or ethics I want to
know like if it adhere to some sort of
you know ethical standard Fair
non-discriminative respectful private
and free from harm um and the last one
the multiplier so this one is the
alignment to what I actually want to get
out of it when I actually use it at the
time and it quantify how well the output
Ally with my specific needs oops sorry I
touched my microphone so anyway so it
acts as a multipliers of the sum of
other comp component meaning that better
alignments will lead to higher trust
score and I know you Engineers here or
developers you're going to be like yep
there's a flaw in this equation yeah
it's supposed to be it's supposed to be
a flaw because I do know that it's so
hard to quantify all of these elements
like you cannot put the number like
precisely on okay um maybe
personalizations one out of 10 how how
personalized this is but I'm putting
this out there so you kind of think
about it and in your own way I'm I'm not
sure in the future you might come up to
me and say I actually find a way to
standardize it let's have a chat because
I would love to know like I'm so curious
like how will we do this but how when I
used this based on my personal
experience I was like yeah in order for
me to trust the output from generative
AI I would kind of think about this as a
fundamental
okay right um actually this is why we
kind of talk about prom engineering
today because you see all of those um
elements prom engineering help increase
one way or the other um the elements of
um the trust model in in generative AI
so PR doing can help you with more
personalization um make sure that it's
ethical um the output is ethical
accurate reliable and transparent and
Ally with the objective you're trying to
use
so who follow this process when they do
prom engineering or kind of like maybe
not prom engineering but say you put a
prom into your um chat gbt gbt to large
language of choice who follow
this no you just you don't use it or you
just put it in and you you just take it
as it is you take okay you raise your
eyebrow at me I think you take it as it
is okay that's fine that's really bold
it's all good pretty much like me when I
um first used chat GPT and send that
respond to my mom hey so usually we kind
of go through this iterative process of
problem engineering so we have a goal
right example here is I need to create
delivery schedule write a prompt I
didn't think about some certain
condition the output comes back and say
hey um here's a plan all like nicely in
a good formatted table and I look at it
and it's like well actually I done work
during weekdays can you move this to
some to to weekdays only sorry I don't
work during weekend did I say weekend or
weekdays I can't remember but anyways
the output comes out and say I have to
work Saturday and Sunday as well it's
like no I don't work on all of these so
move these just for weekdays and then it
uh refy that and in real life I will
send that to my account manager because
they have to know the schedu of when am
I going to interact with my
customers so that's how the iter process
of prom engineering should go it's very
simple but in practice it could be
really curly and it could be um you got
to really think hard about like when you
interact with it um okay so again as I
said trust is a complex phenomenon
calculation equation I propose alone can
describe nuances of Human Experience so
there are some consideration that I have
to think about when I kind of like
experiment with large language
models so number one every word in a
prompt matter don't know if you can see
this is it too small but you probably
can see that I am actually not native
English speaker I'm a second language um
English as a second language speaker so
for me like the article doesn't really
make sense to me my intention was to
make a lot of money I can in 2024 and
there is this phrase where you say make
bank right I I heard my friend say it
all the time make bank so it's like uh
give me a strategy to um you know make a
bank in 2024 and of course it l so down
like as if I'm going to um start my new
I don't know um NAB net bank or I don't
know commonw bank or something like that
but actually just want to you know
strategy to invest and just grow the
money that I have so you see everything
in a prompt matters so it's interesting
as well to see the nuances that it kind
of picked that oh actually make bang and
make a bang is different now second
thing is that risk perceptions influent
trust it depends on you it depends on
how uh your upbringing as well so you
can see there's high risk if the ti is
really high risk right like for example
um you use it for in a medical settings
in um in health care then of course it's
high risk you have less trust you'll be
more skeptical about it if you just use
it for I don't know just some um weird
or practice mathematic like 28 Orange um
trading for apple and stuff like that
but in real life just ask it for fun
then okay low risk whatever it comes out
I'll Trust it I'll be less skeptical
about it or somewhere in between like is
it a bad funu to put a desk in a bedroom
I live in a studio what should I do I
mean it Al also depends on where you're
from for me funu is like okay it's kind
of important like I don't want to work
forever because if you put a desk uh in
your bedroom it means that you're going
to work forever you don't want that
right so it's kind of like depends on
your risk perception that you are going
to trust or not trust um the output from
large language models third thing
subjectivity and objectivities you all
from Sydney right you're from you kind
of from here the majority okay you're
very shy I don't know what can I do to
make you jump up on your share but
anyways that's not what we're trying to
do today you can see
um what's this the most popular cistin
among people in Sydney Thai food or
Italian food what do you think anybody
thinks Thai food yeah Thai food Italian
food okay I think Thai food because I'm
from Thailand Okay I'm from Thailand
definitely Thai food that that is a
subjectivity so in that case if chat GPT
or GPT model comes back to you and say
oh um it's Thai food and you like
Italian food You' be like oh I don't
think so because my personal preference
also if it's objective like 1+ 1 equal
to two you know like the mathematical
problem only have one answer yes of
course when it comes out you might Place
more trust into it because there is no
you can't argue with that right fourth
more trusting in time pressure you see
same promt same promt exactly same promt
what if you have a meeting in 15 minutes
you're going to take whatever because
you don't have time to just think or
Ponder upon it but like if you have all
the time in this world you are just
going to think about oh I need to um
move this around I think this better or
whatever you have time to contemplate
about it okay last consideration
existing knowledge and I have to say you
all really you know have expertise in
your own way and in your own topic now
I'm not a seashell
developers right not net developer
either I'm a solution architect which a
lot of developer hats out I'm sorry um
but anyways say for example I don't know
anything C I know python but I'm working
on a project in C where I need to
perform similar tasks in open AI python
embedding right how can I find a library
for that now whatever that it comes out
from GPT model I'll be like oh I'll I'll
test it I will take it and I'll test it
but for you if you have been doing sea
for a while you might ask like what to
Vector you know there's embedding that
and and you can weigh out the importance
or the pros and cons of its as well like
different options so that's also com to
consideration when kind when you kind of
like be more skeptical or less skeptical
about the output that it's given to
you okay now we're going to go into
promp in techniques I think I kind of
like set the scene enough to make you
understand that when people say they
Love chat GPT or they don't use it
because they skeptical because it cannot
do whatever it can I think it's just
because we just have so many different
views and so many different
considerations as well as our personal
values into um the lens of when we're
using it okay prompt common prompting
techniques this is one of the most
simplest prompting methods giving
instructions AI easily can solve this
right um write the word NDC Sydney
backwards so yes um uh yendis CDN so
that is that sounds like a protocol of
something but that's the simplest way
that you can do with giving instruction
or you can give longer instruction and
it's very exciting to see how modotto
can extrapolate from the instruction
because you can see in here that it kind
of play replace cheap deals with um the
placehold holder for dealership and
salesperson for Jimmy Smith even though
in the example to say hey Dondo chainu
you know placeholder name so it's really
uh exciting to see that um it can have
that level of um transfer learning role
prompting you probably have heard about
this over and over and over but it's one
of the very um effective method to get a
better output from the large language
models so this is you are an X at as a y
or pretend to be or whatever you want to
write it but keyword is the role that
you wanted to play so in this case it's
cost optimization specialist because you
know what this seemed to be a very
common topic amongst all my customer
these day how do we optimiz cost in
Cloud so just list it out and um make
make a uh comprehensive strategy for for
it um the next one is f shot prompting
anybody have heard about f f shot
prompting yeah you have heard about F
shot prompting would you like to come up
and no okay that's fine I'm sorry um so
a common way to adapt language model to
new tasks is through F shot learning
what do they mean by that you give an
example few shots is whatever shot that
is is an example that you give it for it
to learn it's a set of uh training
example that you provide so that you
give a context to the model what uh I
think Scott kind of mentioned the
context a lot in his keynote this
morning so the context to the models now
I'll tell I'll show you the graph in the
in the couple of slides I'll tell you
why it is important to have the short
prompting then you also have other shot
prompting you have zero shot prompting
right in English what does this um
Mandarin character mean right so this is
the most basic form of prompting you
don't give any example for it at all
it's simply showing the model without
any um example or generating any
response just completion you don't give
an example so in English how uh sorry
niow is a man in Chinese greeting
translate to hi or hello and you have
oneshot prompt so onot prompt how many
example one one shot prompt if you
listen to me you got to pay attention
you know because you have to help me
improve the promp
later how
many one okay yeah that's right so one
shot prompting is when you show a single
example so zero shot um is where the
model make prediction with additional
context right but one shot or few shot
prompting use small amount of data or
small amount of that context that you
give usually between two and five good
numbers um it learns from the limited
data and use to generate uh response and
makees predictions for a similar a
similar task or prompts so these shots
uh prompting it use transfer learning
techniques and it kind of trying to
adapt to whatever task that you ask it
at that point in time it's very good for
smaller model for performance so GPT 4
you know it's brilliant capability is
brilliant but it's so expensive right
it's so expensive if you want to use
like a local models or open source this
might be uh less capable but it's still
doing the task so you can actually use
the uh short promptings to help it
learns in context um so maybe consider
that this is actually from um a paper
called uh language model are few shot
Learners you can see that as we increase
from zero shots actually maybe this side
as you increase from zero shot to one
shot to few shot um especially for GPT
models uh three the accuracy go up
significantly so smaller model really uh
benefit from you having more context
more example into
it okay so then you can also combine all
the technique I talk about before right
you have the role you have the
instruction you have the example and in
this case you can see that okay your
social media analizer want to classify
this you kind of give uh the role and
you give the instruction what it needs
to do you give an example of the two
tweets before and then the last tweet is
like it needs to complete what is it and
then it kind of learn have that transfer
learning capability to pick it up that
oh okay we uh categorize um the topic
here
okay style and
tone you know how I talk about the
personalizations so say I have a new
grat High joining don't know if you can
see it can you see at the back is it too
small yeah okay so you
know have a gr High joining Diamond Corp
right and you see like the first one
without style it say dear new High's
name we think we are through to welcome
you blah blah blah and then you have
your name human resource manager at
Diamond Corp whereas when you say witht
style and I know that this person just
graduated from University and genz and
you see like how it's actually give the
output out uh give the respond back it's
tailor to gen Z it's like hey hi and I
know that because I'm gen Z actually I'm
in between gen Z and Gen Y and I was
like yeah I feel so included in um the
company uh onboarding experience
but that is something that you can do so
there's some um
some examples of the style and tone that
you can be more purative professional
narrative formal descriptive and so on
all the adjective you want just put it
in okay and you also have the format um
the format could be a table could be a
save could be code block report tweet
well code block is really depends on
what you're trying to do if you kind of
go into GitHub co-pilot it's kind of set
it as you know code block anyways um but
depends on which API you call and which
um GPT tool you use as well um could be
tweet video script XML Json bullet
points um and so
on anybody actually have a a weirder
format than this can you share I would
love to learn more have you used all the
formats than this one that I put on the
screen no okay that's fine if you when
you shy come and talk to me later okay
all right so let's move on to a more
advanced technique have anybody heard
about Chain of Thought
prompting
oops my boss I'm
sorry
wait I'm
sorry okay Chain of Thought prompting
not many of you have heard about Chain
of Thought So basically this technique
is really good for reasoning the complex
task
this can be combined with f shot um
prompting to get better results on more
you know logical tasks that it require
reasoning before responding so this
method is very effective for complex
task that's me um hit my 30 minutes so I
know that I need to speed up so this
method is really effective for complex
tasks that can't be solved in one goal
so you have to uh divide in a smaller
steps and you kind of linked it together
so you can see if it's just one shot
prompting um as in general you can see
that Roger has five tennis ball he buys
two more um of tennis two more cans of
tennis ball each can has three tennis
ball how many do we have now and you
kind of say the answer is 11 you give it
like an example the answer is 11 and you
can see like the when when you ask it to
complete the answer is wrong the answer
is 27 which is not 2 uh s it's it it
should be um something else Shain of
thought prompting is when you kind of
like walk it step by step instead of you
say thinking step by step you kind of
show it the scaffolding of okay what's
going to happen next so in this case is
Roger start with five balls two cans of
tennis um
each two cans of tennis ball each is six
tennis balls 5 + 6 equal to 11 and so on
and when it kind of give you the respond
back it walk you through the thinking
process behind the scene or the
calculation process behind the scene
if you use gbt 4 today it's there for
you but it's also expensive it's I think
it's premium so you have to pay or you
have to use uh Microsoft co-pilot I'm
not sure if um if I think you have to
have Microsoft account or something like
that but um it depends on the tool that
you use and the
models okay so that's Chain of Thought
now we also have Tre of thoughts so in a
typical Chain of Thought situation the
model will come up with answer step by
step right right create the next token
based on what they just written but
without going back to fix any mistakes
it's keep making up words it keep having
more errors right the more errors uh is
made the more um it's go down um the
generation so it's hard for the motor to
correct and move the uh move it forward
um now tree of thought is similar to
Shain of thought but essentially just
before you kind of um get the output
back you kind of walk it through each
step of the way so you see actually I
sent this to my boss as well so I said
you can use this prompt to help plan
your resources and he was laughing and
he was like do you mean our team I was
like well I don't know but anyway so I
had a team of 10 Engineers 100 customers
um the original prompt that I uh the the
one that I got and you can test it for
yourself the one prop uh one shot
prompting without example you can see um
that the the answer is actually still 10
customer each right because it get
confused so if you kind of use the tree
of thought prompting it helps um with
the calculation or the reasoning behind
the scenes so how 10 uh I have a team of
10 engineers and 100 customers they had
10 customer each the company laid three
of them off which left me with seven
Engineers you kind of help it do like
the calculations at each step soon after
that to resigned to build their own
business which left me with five
engineers and I can backfill one
position would bring me up to six how
many customer should I reassign to all
of them now and if you test this out and
it will tell you I think it's like eight
or six or something like that I'm not
good at maths today but let me know what
you
get okay and then system message which
you probably have seen a little bit
behind the scene as well if you have not
used um the system message
before this one is a um the example that
I took it from you know the GPT um the
GPT that you can build on openai the
chat GPT GPT this is the Sapia um uh
connection to Gmail so their system
message is like this because I was
trying to call something from my Gmail
Gmail account and summarize it or like
pick up a keyword or something like that
so the system message it doesn't have to
always be like the role promt it could
be um mentioning about the step that it
has to take the step that you need to
verify first um and sometimes you might
hear a system message uh or they refer
to meta promp so like the prompt that is
not shown to the user which is like
behind the scene um so you can Define it
however you like you can ask the model
to complete the task you can Define the
scope and limitation posture tones
language syntax and all those things
um another example oops
sorry you know what I was looking at the
wrong slides I'm sorry the sapio one is
here the second example I'm so sorry
about that so this is the sapure um this
is the sapure example so it kind of tell
the custom actions and the rules and the
function call it has to do um the
previous one is example of you know um
the gating process can we see okay like
to avoid copyright infringement if user
use copyrighted you must you must not
violate any copyrights at all so
actually I don't know if you know about
this but this Clause have to be in your
AI um in your AI uh application in order
to be protected uh with the copyrights
that we the copyright commitment from
Microsoft because we kind of have that
um copyright commitments coming out for
Azure open AI as well but you have to
put like this Clause into to your meta
promp in order like as part of the
requirements to be protected for
that okay and then you also have promp
shaning um so promp shaning you break
down the multiple uh tasks into a
smaller promts and you pass the output
to one prom to the next you can change
three different prompts together and you
can see it could be like three different
tasks um but you use like a variable to
refer to the output of um the previous
one so this is good for multiple steps
invol different tasks um it can also
help you debug or troubleshoot
especially if it's uh if the task really
complex or really long so in this sense
I actually have an example here let me
see can I see this okay can we see
this okay so you see here's the system
message right you are an a AI power
medical assistance user want to talk
about specific patient clinical note and
and then um you will convert this note
into a referring doctor to a referred
doctor and this is a variable and you
can set the variable in here so I kind
of set the referred doctor meridi gray
referring doctor Shan Murphy and I have
patient email because I want this to
Output a patient email right uh and send
it to John at brown.com so patient
clinical note let me see if I have
okay let me I have a patient clinical
note like a mocking one here and I'll
just paste it in and you can see that it
say okay John Brown so usually doctor
would just write gibberish or like quick
keywords 60 old um 60 years old male
symptoms St elevation I can't pronounce
that word I'm sorry
mord
myocardial infection or something like
that so okay I'm going to apply change
and I'll see that
um and let's say I'm not just a chat I'm
just trying to do something else I'm
going to
say every time that this is called right
automate automate
um the
letter for John Brown and you can see
that you can hook this up to your um
database and stuff like that but in this
one I'm just going to use the simplest
um chat-like experience for us to see so
I'm going to choose the deployment
models I have 3.5 turbo um six 16,000
contexts um four and 32 um thousand
contexts as well but I'm just going to
use like the cheaper one because I have
limited credits in my subscription so
I'm just going to say okay
automate and then from here just like a
little you know do points it kind of
automate um the email write the subject
and dear Dr merited Gray so that is what
did I put refer doctor right okay so Dr
merited Gray need to see John Brown
after this yada yada Y and then sign off
your name obviously I did not write this
promp well enough so then it's if you
can you can also tell it in this prompt
uh system message to sign off with your
name so that how you you would do prompt
chaining when you have multiple tasks
okay all right now a little bit more of
advanced technique when you adjust the
parameters you can also see the
temperatures um more more numbers um
depending on the API you call the zero
and two I think on open AI if you go
into the playground if you go on aure
open AI it's only up to one so um zero
mean less creative conservative uh
predictable
two more random that mag Magnum Magnum
or magma whoa was it
magum I can't even say the words
properly but you know what I mean to
refer to um Scott session this morning
but you know it's more out of control
more creative more unpredictable but
that also depends on um how you view
creativity um so yeah if you lower the
temperature it will give you more
conservative more focused response and
imagine that would be good for say if I
were to write a referral do uh Letter to
Dr meridi Gray I don't want it to be
creative right I wanted to tell it as it
is okay next one as well is top K and
top P have anybody seen top K and top PE
before depending on the API you call
sometime top pe's come up sometimes top
K top case come up top piece kind of
always there but essentially it's how
the model pick out the output token so
top top K is basically ranking so if you
give it three it will pick like the
three the three words right in on the
list the high the highest top here is by
percentage So when you say 0.7 it's just
going to look at all the sums of all the
uh percentages and then it would say
like oh okay um if I only one 0.7 I'm
just going to get the four out the list
because if you make a calculation you
see it's all of that equal to Z
70.
35% so that's how they kind of pick up
the words but this is more of um trying
to make your uh the words or the outputs
more creative or more varies but it also
depends on the application that you are
building for as
well okay frequency penalty and present
penalty um essentially they trying to
control the repetition of words and
phrases that it responds so for
frequency penalty the higher the value
it discouraged the model from generating
uh the same word for multiple times more
varieties in the output um maybe use for
Creative content writing avoid
redundancy anybody use co-pilot today
maybe like just the normal Microsoft 365
co-pilot okay you see like there's like
a button that you can see like more
creative or more professional or
something like that so that is how they
control um how the outputs are going to
be and you also have prent penalty so PR
penalty um this one control how relevant
the response is so the higher the number
it encourage the model to generate words
and phrases um that already presents in
the input prompt so if you talk about
Apple in your prompt it we trying to
stay on the topic bring more Apple in
but on the other hand if it's lower it
means that it's less constrained by the
input prompt so it will kind of penalize
on okay I don't want to see apple again
so maybe um some some other new word or
some other new um tokens and that make
it well it depends what on what you're
trying to do you if you wanted to
introduce no new topic um or go off a
tangent you can do that but if you want
to to focus on a topic you um have to
increase the the um person
panalty
okay anybody have heard about retrieval
augmented
generation yes okay that's good um so
basically you're trying to ground the
models on something right because you
don't want it to hallucinate which is a
cute nickname that we give it to large
language model when it trying to make
stuff up now
personally I don't think hallucination
and just my personal opinions I I don't
think Hallucination is there's something
wrong with that it's like it's not a bug
it's not something that is wrong it
depends on what you're trying to do if
we go back to setting up your objective
of you're trying to do with your uh lash
language model prompts then if you want
it to be more creative of course I want
it to hallucinate it a bit like I don't
want it to sounds like a prime minister
of um Thailand or something so then you
would want it to hallucinate right but
this is depends but for retrieval
augmented
Generations um usually we want the
groundedness we want the cations of the
sources when you ask as a uh some sort
of knowledge look up you want it to go
fetch that information returns it in a
way that um human like and that it give
you the source to that um PDF page or I
don't know some CSV uh C CSV files so
rag techniqu solved this problem because
it's ground the mod Els and you can see
that um usually you have the user prompt
and you might say help me assess the
enironment impact in X and you might
already have your data about
environmental impact somewhere in your
organizations right so you will put this
um you have your cury data and then it
will get sent right it get embed first
get sent to um the search index so the
search index is a is a separate entity
because you also have to index your
documentation before you can call it
back um to combine with with your
context and prompt so you remember the
system message I show you you can um
insert the context in the system prom
and sometime the context could be 500
pages right but it won't be included
until you call it and then um you
combine those it's query the model uh
when you say query the model it send um
the request to the endpoint um the
whatever the model you use send that to
that uh endpoint and then give the
output back
let's have a look at it quickly because
I kind of have this what time is it what
time do I have
until uh 40 okay so 10 minutes yeah okay
that's good um so then if I go
to okay this is another example um is it
too small hopefully it's not too small
anybody used promp flow before L chain
some FR okay you use a bit Lang change
um so essentially some you see how when
we want to do prompt chaining or rack
pattern or something like that it's
there's so many workflow that goes into
it right so either use open source like
Lang change um that you know all code
based and you have to um put it all
together or you can also use prompt flow
on azro AI Studio as well which I use
because um I love when things are kind
of easy um I have to admit and it
actually helped me see how the flow
works so the flow that I show you on the
presentation before you can see that
there's an input in here that goes in
you have to modifi uh modify the query
with history is it too small but I'm
what I'm going to show you is actually
the
index oh search index I think it's here
so as part part as part part of this is
trying to search on the question from
the index document I already index my
document which is this is the document
that I put into my blog Tik Tok what's
next Trend okay uh which is where's my
Tik Tok PDF okay here here so there's
this trend Tik Tok what's next Trend I
want to learn how to um create content
according to 2024 right so I have that
in my blob and I index it and I I can
see indexes and I set this up and then I
hook it into my flow and I call it in
and this is um where the UI of The Blob
storage is and the query and somewhere
in here I have the Tik
Tok it's really hard to
see okay so this is not going well for
me but that's okay but you get the gist
I call the index from the PDF file that
I store on my blob storage right now I
have the runtime up I can chat with it
and it's also keep the um chat history
as well so I kind of test this
beforehand because I don't want it to um
not work while I'm here so let's
say I'll show you this so
help me tailor my content to Tik Tok
Trend in
2024 and it will go to the
documentation and I can uh share this
with you because curiosity Peak is a
particular word that it kind of used in
the Tik Tok what's what's next train
which is here where is curiosity
Peak curios City
Peak what or something like that
breakthrough
oops our brains not supposed to work
with two screen I'll let I'll tell you
that so okay the Curiosity Peak it's
like a theme for this 2020 um 4 sort of
thing and it was give me all these
advice and I say I want to talk about Ai
and say great topic um this is how you
can adapt the trend to how you're going
to present AI on Tik Tok and in here you
can see as well that it site the sources
what's next for 2024 Trend report and
you might ask me like then where do I
put my prompt so the all the prompts
informations like all the system prom
system messages are in here so you are
an AI enable social media expert blah
blah blah you have this context so this
context is the one that got sent from
the index so if I go in here I could see
like the
output it has the system message and
combine with the context that I call
from Key finding 2024 the Tik Tok Trend
okay so this is how you would do um the
r pattern um in uh for with the lar
language model so in a way you still
have to do the indexing of your
documentation because it can trying to
find the semantic meanings of it okay
okay I know that we almost at time we
have 10 minutes but I really want to
show you patterns I know that we not
going to get through all of them but you
know pom patterns how it's similar to
software patterns and you are software
developers and engineer you kind of seen
it before so it kind of offer a reusable
solution for specific problems um more
specifically um it codify the approach
of solving a common challenges so the
researchers I think white AAL um he kind
of comes up with all of these different
patterns um that we can use and is
really helpful um please take a
screenshot or something like that not
screenshot please take a
um photo so you can have a look at
yourself later or I can send it to you
after um the talk as well but
essentially for example you can see
this this is a what we call a meta
language creation sometimes you want um
to use a symbol to do something for
example when I say x I mean y um an
example of this is when you see the
triple um exclamation exclamation mark
in a sentence return a message with a
bless Emoji to remind me to calm down
right so for example if I go to here and
I
say if I put my system message in here
and say you are encouraging a assistance
when user put um this exclamation mark
always include the bless emoi in your
prompt apply
change okay and then I'm going to
say okay what do you want to say
anybody what what do you want to put
into um the chat
anything that's a thing I'm standing in
between you and lunch and I know you're
all going to get very angry soon is it
time for
lunch
yet why is she
still
talking okay all right let's see what it
say hello it is understandable that you
might feel impatient bless remember to
take a deep breath and remain
calm okay so you can see how that kind
of like really really useful right in
the in terms of meta language Creations
okay there's a more this game play like
you know create um security game and it
turned that into a Linux and you can
like automate how you want to be
challenge what whatever the topic and
there's more but because
we are running out of time so I am going
to skip through the best practice so
quickly best practice you know sometime
you put everything all in at once um
it's really good and more effective to
kind of have the triple hash or like the
triple
quotes and you also have the um
specificity um you can't just say hey
write about something a poem about what
your open AI but you have to be more
specific of what you're trying to get it
alph from it so write a short inspiring
poem focus on the future of work like
what is the Focus right and um
articulate the desire output um through
examples so it's kind of more effective
um if you trying to um give it an
example as I said the few shots or shot
prompts um five to three sentence um
to tell what you wanted to do right
reduce the fluffiness and imp position
in the description you can only see that
every word in prompt matter because you
know make bang and make a bang is
totally the different things uh more
effective um instead well don't say what
not to do kind of tell it what to do
instead because we get confused as well
when we're trying to say negative and
negative and then trying to be positive
so you know same thing as um uh the
language models are leading words cuz
sometime you use it for coding and
sometime you might want to use it it for
a specific language okay before before
you go to promt because I kind of
promise that I'm going to do a live
prompt for or or you help me improve but
I don't know yet if you kind of like
would will be less shy about it until
now because we have like what five
minutes left well what's the last prompt
you
use if you can tell me I can help you
improve everybody here can help you
improve or maybe it's too personal like
a search results
yes
Michelle
help okay okay all right let's do this
so how do you can you tell me exactly
what what you you word
help I
said I said uh please can you write a
LinkedIn recommendation for a prote who
used to be a student learn Ambassador
helping me Teach students in the
Microsoft accelerator
program ambassador to help to help me
Teach students in the Microsoft student
accelerator program okay Ambassador I
cannot spell that but that's okay still
learn ambassador to help
me Teach students other students
in
Microsoft learn
accelerator
program program okay let's do this let's
see what it's going to give it to us
okay dear oh wow I'm writing LinkedIn
recommend wow who write LinkedIn
recommendation like
that this is like an email to somebody
high up how would we do
this how would we changes right LinkedIn
recommendations what is your goal
Michelle are you putting this somewhere
yeah it's in his LinkedIn
recommendations but I used um Bing
co-pilot to do it and it did a really
good job I thought it was long but I
pasted it in it looked good okay so um
okay program the let's say the output
okay actually maybe we changed the
system message how about that you are
LinkedIn
expert oh nobody helping me here we're
going to go out for lunch
no okay you are LinkedIn
expert you
help user
refine by the way I didn't know that
Michelle going to say this okay this is
totally unstage so I put my brain is
blank help me please so you help user
refine
recommendation dations what's the goal
of
recommendation when you write a
recommendation for somebody what do you
want them to get out from it a job okay
recommendation uh for a job okay user
recommendation um you help youer refine
recommendation for them to get a job for
them
to get a
job to get a job or more job
opportunities or be more visible for job
opportunities be more visible for job
opportunities what job is
that any job cleaning
developers developers
okay uh for Developer jobs
opportunity
okay what else can we make this better
so that Michelle can kind of use
it for her LinkedIn
ambass a
style okay okay uh I say you will
remain
professional concise and concise I like
that and
concise and concise okay
here okay okay you will remain
professional and
concise oops sorry um stop so yep one
second give me two more minutes please
so remains professional and concise not
more than two sentences oh two sent or
two paragraphs
sorry
[Music]
pardon we can change it to
speeches you um you teach teachers
teacher
where is it oh you will remain
professional concise of more than two
paragraphs sorry one second my brain
cannot multitask
teachers oh okay to get a job the
okay for student learn ambassador to
help me Teach other student where should
I put the word
teacher along with the users okay users
and teachers to help user and teacher I
see to help the second line uh you are
LinkedIn expert you help users and
teachers and user okay all right we
happy with
this that's all good I'm going to save
it first and then uh ground what student
learn and bassinder is like um
insert insert factual information of
what what a student learn Ambassador is
make sure you no no no no like go get
what a student learn Ambassador is
ground it okay we will pretend we will
pretend that that happens already okay
because it does take time to set it up a
little bit as in like yes that's true so
potet hypothetically that is really just
like copy and paste it Google it copy
and paste it okay okay oh my God you're
so smart Microsoft student learn
you know when you stress and your finger
is no longer with you that's what
happened right now okay just one
second okay Microsoft
become Michelle where is the description
of uh student
learn okay okay what is
student learn ambassador
okay okay okay okay that's smart all
right hang on one second let let it let
it do it
things see
more oh come on stop responding oh good
they all
hungry okay let's go
back that's the doctor one okay and we
can say maybe we add variables and we
say context how about
that
context definitely not
context okay and then
paragraph
used
context okay for your reference
why can't I type use
reference use context for your
reference
for MSA okay oh well in fact let's just
type it out Microsoft
student
learn
Ambassador okay all right we happy with
that
okay look like I Al so not it's so tiny
I'm so sorry like it's so tiny on this
screen
continue okay now I
say so what's the name of the student
Michelle okay okay sorry I I will
probably use something else okay one
second uh write
me a
LinkedIn
recommendation
for okay okay for Gia who has been
a
Microsoft
Ambassador okay what else do we missed
nothing clear theob to get a job you are
very smart that is really good okay we
will clear the
chat all right let's
see how does this look is it better
better than a long
email great okay I highly recommend jar
for any job opportunity yes that's right
I high employability
um she been passionate about all of
these things yada yada y so would you
trust this more than the previous
example yeah okay so I know this so
simple like this so simple but this
would be like the actual thought process
that kind of go in well you do more
iteration if you want it to be more
creative whatever you change the
parameters here as well okay but that is
how you would kind of um go through the
prom engineering and of course you can
incorporate like change of thought Tre
uh tree of thoughts and all those things
that I showed you before today too um
okay um before the last slides all right
this is a homework homework for
you help me improve this prompt if you
ever well take a photo because it's
going to be homework so if you take a
photo if you improve this prompt send it
to me and you know let's you know talk
about it toor me on cloud computing
topics I need to sit this a one or two
exam certification soon okay need to
make it interactive rigorous inclusive
and well paac okay and you connect with
me on LinkedIn and then we can talk
about it if how's your prompts uh going
anyway so that bring me to the last one
so can we really trust the large
language output yes we can um but trust
thoughtfully because it also depends on
on you know your personal
predispositions the the risk you
perceive um how what's the task you're
trying to do at the
time it depends right so no matter how
good of a prompt you can design you
always need to check and never May uh
never take the output as it is free from
bias um or uh any discriminations or um
um uh factual so anyways um I hope you
have fun I know you all look very fun
and I'm standing in between your lunch
which I'm quite late to finish this uh
session thank you very much uh for
sticking around and I will see outside
[Music]