computer what time is it Friday
15:25 computer what's the weather 39.4
5° computer tell me a joke about Lewis
Rossman and Bill
Clinton why did Lewis Rossman and Bill
Clinton start a repair shop together
because they both know a thing or two
about fixing crack screens and political
scandals computer turn light
on light is on computer turn light
off light is
off welcome back as you know I am Eli
the computer guy and in this session I'm
going to show you how to build a voice
controlled app using python that will
also be able to respond to you in the
English language so basically we have a
python script here I am able to speak to
the computer the computer parses what it
is that I said then based off of what I
say it then triggers different functions
to fire off uh and then it gives me some
type of verbal response uh it'll tell me
what the weather is it'll tell me what
the time is it will tell me a joke and
when it turns a light on or off it will
tell me that at least thinks that it's
turning the light on or off and we were
able to do all of this with python so
one of the important things that I try
to get across to new students is that
when they look at the world of
technology they see all of these very
complex subjects and complex
Technologies and they think oh my God
how am I ever going to be able to do X Y
or Z how am I ever going to be able to
build artificial intelligence how am I
going to be able to build some kind of
voice recognition system how am I going
to build the next GPS and what I keep
trying to tell all the youngans is
remember AI is hard API are not and
frankly for most of you you're going to
be dealing with the apis you do not have
to try to figure out how to build the
next open AI or CH GPT or whatever else
you have to figure out how to make an
API call to get the access to the
resources that those platforms are
providing uh so that you can then bring
them into your code and so one of the
really interesting things nowadays is
you see these really complex
Technologies of things uh again such as
voice recognition it's like wow the
amazing part in the modern world of
technology is for voice recognition in
your python code you literally install a
couple of modules and write a little bit
of code uh you have to install a module
called Pi audio that allows you to be
able to actually uh pull in audio from
your microphone uh you then uh install
another uh module called speech
recognition or package called speech
recognition in there there's a there's
an option for Google uh that's what
we're going to be using today and that's
basically it Pi audio allows speech
recognition to be able to get the uh the
audio that's coming in from the
microphone speech recognition then takes
that audio turns it into asky text and
then once we have asky text we can just
deal with it really really easy again
and that's one of the one of the things
to to really be considering in the
modern world when we're using these
computers is basically can you take
complex information and somehow
turn it into text can can I parse a
picture no that's very difficult can I
send a picture up to Microsoft's Vision
Services it responds with text in a Json
format and then can I parse text in a
Json format oh yeah I can do that in
about five minutes that's incredibly
easy and so that's one of the things
that that I want you to take away uh
today from this this project that I'm
going to be showing you is basically
this idea is that you can use these
modules you can use these services that
have already been created basically with
that you can turn things and to to just
generic text once you turn it into text
you can parse it just like it's generic
Tex textt because it is generic text and
then you can fire off events to happen
and what you find out it's actually
really pretty darn simple again AI is
hard apis are not and so in this class
today we're going to go through the
script I'm going to show you how it
works uh we're going to be dealing with
the operating system so we use the OS
module that's how we get the time we're
going to deal with a rest API so we're
going to go out to a couple of uh rest
apis in order to get what our current
location is and then to get the
temperature at our current location uh
we are going to use open AI API uh for
that whole joke functionality if you
want to tell a joke we going to use open
API in order to do that uh and then for
turning the light on or off we're going
to use something called the serial
module the serial module is going to
communicate with our nice little Arduino
down here the Arduino is connected to
what's called a relay the relay is
connected to our light and that is what
turns the light on or off and so I'm
going to show you how all of this works
again to give you the idea of not just
how this particular project works but
how you can build something yourself now
the core to today's project is a module
called speech recognition as I'm doing
their cording they're at version
3.10.1 uh so basically you can use pip
or pip 3 install speech recognition and
this will then install this package uh
onto your system so that you can do the
use the module one of the nice Parts
with speech recognition is that you can
actually use a number of different
speech services so you you will have
heard me talk about in the past
something
calleda or service oriented architecture
right this is an architecture style uh
that really became prominent about 15
years ago the idea being is that you
should be able to write your code you
should be able to design your platform
and whatever functionality that that
platform or that code needs you should
be able to swap it out uh relatively
easily right so if you're using twilio
so twilio is something for SMS messages
basically text messages so let's say
you're using twillo within your
application and for whatever reason you
need to stop using twilio maybe your
Paypal account will no longer send money
to them or twio they they they jacked up
the price and it's just too expensive
for you whatever else in service
oriented architecture the idea is you
should be able to go out to another SMS
API provider uh find the one that you
want and basically with a number of
lines of code just be able to then you
know paste that into your code so that
you're Now using that new provider uh
instead of twilio the idea being all of
these services uh you should be able to
mix and match and change around uh with
re relatively uh few problems uh with
your system when you do it you're not
going to have to do massive migrations
or anything like that one of the
interesting things with the speech
recognition module is that you see how
many options you have in the modern
world for speech recognition right uh
you have SP thinks you have Google
speech recognition you have a Google
Cloud speech API with Azure speech Bing
voice houndify IBM spee snowboy
tensorflow vosque open AI whisper and
Whisper API these are all different
voice services that will take your voice
and turn it into text now there are good
parts and bad parts for some of these uh
some of these actually are able to work
offline uh some of these are going to
cost you a little bit more money some of
these are going to be part of a larger
Cloud infrastructure that might be
useful for you again if you're already
in the Microsoft World using something
like azures speech services might make
sense because you're already in the
Azure world right the idea being is you
pick the best speech service for what
you know your particular uh situation so
these are these are a number of
different options here and you can use
these options uh with the speech
recognition package uh but we are going
to be simply using Google speech
recognition today now there's a couple
of things to keep in mind when you do
this one it is an API call so with these
uh these this vosque down here an open
AI whisper that works offline these are
offline which means it just runs like in
your computer and you do not need uh
internet access for those to function um
so if you're using something that's
online right you do have to have an
internet connection for this to function
if my internet connection goes down that
script that I show you will not work the
other interesting thing with this is
with speech recognition I like this
because it's very easy to get up and
running again I like to show you folks
easy stuff you can make your life more
complicated right well the nice part
with this is you actually do not need an
API key so for whatever reason Google
has put out an open Global API key uh
that this this module is able to use
right so basically it's making an API
call and it is using use is using an
official API key to make that call but
Google has just made that available and
made that open so people can play with
the service again that's what a lot of
these companies do you know they they
make it free or incredibly inexpensive
for people to play with something and
then as soon as you put it into a
production environment you find out
exactly how much this crap cost one of
the warnings that I will give you with
this though is that we don't know what
the B what the throttling of this is we
don't know what the limits are I haven't
run into limits I've actually been using
this for a couple of years now I never
seemed as far as I know seemed to have
run into many major issues using Google
speech recognition with with that basic
API key uh that that's provided with
this in this module but again if you're
going to put this in the production
environment you're going to start
hammering the hell out of the system
just keep in mind you may hit some
throttling you may hit some constraints
because they don't they don't want you
using this in a production environment
for free so that's one of the things so
basically this is how we're going to
turn the speech into text then when we
want uh our computer to be able to talk
to us the other service that we're going
to use or the other package that we're
going to use it's called gtts basically
this is Google's text to speech service
and with this we're going to feed text
to gtts it is going to turn that text
into an MP3 file and then we're actually
just simply going to play that MP3 file
and that's what's going to get our
computer talking to to us again this is
uses some weird API key that keeps
working so it seems to work fine but you
know if you run into issues just keep in
mind right you might actually have to
pay for a real API key or something like
that one of the important things with
this class is again I don't want this is
not about building something that's
going to go into a production
environment this is showing you how this
works so that once you build something
and once you understand how this works
then you can figure F out the particular
Services the particular API calls that
you want to make in order to to flesh
out whatever it is whatever project it
is that you want to do so anyways that's
the uh we have speech recognition and we
have gtts and this these are the core
components of the project today so now
let's look at some snarly code again I
will warn everybody at home trigger
warning trigger warning I code like an
mcse an NT
4.0 and anybody who's been around for a
while they understand that joke anyways
everything that I'm going to be showing
you when I build these things do realize
I'm focused on trying to present
Concepts to you actually show you stuff
that works um some of the syntax and the
formatting I probably wouldn't do in the
real world this way just because it
might look a little ugly but anyways
this this works it'll it'll show you the
concept um I'm not sure I'd copy and
paste and give it to your boss you might
want to rewrite it a little bit if
you're going to kind of try to copy this
at home so anyways uh up here we have
our shebang so if we just want to be
able to run the script without calling
Python 3 we have the shabang up here uh
then the first thing that we're going to
import is a speech recognition module uh
so we're going to have to install the
package for speech recognition so pip 3
install uh speech recognition but that's
all actually one word speech recognition
is one word but that Imports the speech
uncore recognition module it's just how
it goes right uh do remember again if
you look at the documentation for this
to be able to import audio from the
microphone you do also need to install
Pi audio now Pi audio installing Pi
audio you know that might be its own
little troubleshooting routine I've
installed Pi audio onto uh what I'm
currently using here this is actually a
2018 Intel MacBook Pro I've installed it
on here I've installed it onto the M2
MacBook Pro I've installed it on the
Linux boxes installing Pi audio onto
different systems can be quirky can be
quirky so anyways uh that is probably
going to be the most difficult thing for
you if you do pip three install Pi audio
and it works properly or you do pseudo
app install
um Python 3 hyphen Pi audio I think it
is uh and it goes well then you're doing
good um you might have to troubleshoot
some stuff I will just put that out
there again that's one of the
difficulties with with trying to teach
this stuff online is again we're dealing
with all kinds of different systems all
kinds of different configurations
something as simple as installing a
module can be a shocking pain in the
butt so anyways Pi
audio that's that's gon to be your most
all thing for today you're going to have
to figure that one out on Google anyway
so we have Pi audio installed in order
to be able to get the microphone then we
have speech recognition installed so
that we can get speech recognition so
this is the overall uh name of the
module so we're going to use an alias
right an alias in Python so as Sr just
to make our life easier when we have to
type this out from UND uh lowercase gtts
we're going to import lowercase G
uppercase TTS right so basically from
Google text to talk service we're going
to import this so that we can you know
turn text into speech import OS this is
what's going to allow us to interact
with the operating system from date time
import date time this is what's going to
be able to give us the time for when we
ask for time import requests so we're
going to be doing uh rest API calls
we're going to be going out basically to
websites being able to pull information
in so that we can do something with that
we're going to need the request module
uh then we're going to import the serial
module so the serial module is what
allows us to the serial communication to
our Arduino so we have that here uh then
we come down we're also going to import
uh open AI so when we do the joke you
know we ask a joke about leis Rossman
and Bill Clinton uh we're going to be
using open AI for that then we're going
to come down here we're going to have
the open AI API key I'm not going to
show you my API key today but that'll
get plugged in here so you either plug
it in here or you'll plug it in as an
environment variable
H the secure way to do it is to plug it
in as an environment variable uh there's
some documentation how to do that in
open AI um I do it this way just because
it's easier to show you folks how this
is working R is going to equal sr.
recognizer so basically uh the
recognizer um function within the speech
recognition module we're going to be
we're going to call that using R in the
future language is going to equal
English uh then we're going to come down
here for the serial connection so
serial. serial so this going to be the
serial connection and this is how we're
going to be connecting to the Arduino uh
so with my particular Mac this is the
line SL
decu uh. USB modem
14101 right um so if you are using a Mac
you go to the dev directory you you look
for cu. USB modem and then you plug in
whatever that is and this should work
for you uh if you're using Linux uh this
line would be the
TTY AMC Z or one uh it's slightly
different in in the Ubuntu Linux world
if you're using Windows I don't know
what it is you may have to do a Google
search uh for basically serial
connection uh between uh your computer
and the Arduino we set this at 9600 bod
so that so basically the Arduino is
configured to to be um uh listening at a
certain speed so we have to make sure
the speed is the same time out after 1
second if it doesn't connect for some
reason and then we're going to flush so
basically we're just going to flush uh
all the communication to make sure we
don't send erroneous things to our
serial connection then what we're going
to do go do is we have these are our
functions so we have a g time function
we have a get weather function we have a
get joke function and we have a switch
light function so basically uh when we
say something one of these functions
will be called so these functions are
going to be before the while true Loop
so normally when you're running uh
speech recognition it will run once
it'll listen for whatever it is you have
to say and then it'll stop so what we're
going to do is we're going to do a while
true loop on somebody else's API key
again one of the warnings in the real
world in the real world be careful while
true Loops when you have to pay for the
API key right so every time you make a
request using an API key generally it
will cost you know a penny or two right
if you do a while true loop it's going
to continuously Hammer that API service
you know you basically you're paying a
penny or two literally as fast as this
thing can Loop that can get expensive
really quickly so with none of this with
none of this am I actually paying for
anything so I'm not not too worried
about that while true Loop so anyways
basically what this is goingon to do is
it's going to to try to listen on the
microphone uh if it hears something
it'll process it if not it'll Loop if it
hears something it'll loop loop loop
loop loop so whether I hear something or
not it will just continuously Loop and
it won't stop until I tell it dismissed
uh essentially uh so we come down here
we're going to have a variable called
response and we're going to set it to
nothing we're going to have a variable
called phrase and we're going to set it
to nothing again this is also looping so
one it creates and sets these variables
at the first time but also as these
variable values get set every time we
Loop through we want to clear out the
variable values so that we're starting
with something fresh we don't want we
don't want the exact same request again
to start looping as fast as this thing
can possibly Loop anyways so we're going
to do this so with Sr so speech
recognition. the microphone function and
since we have Pi audio we can read from
the microphone as Source we're going to
print out on the screen say something on
audio is going to equal r. listen to
Source right so sr. microphone as Source
r. listen so up here we have Sr R equals
sr. recognizer so basically what is
going to be happening here is recognizer
is going to be listening to the
microphone and saving that to the
variable audio now the important thing
to understand with this audio is at this
point point the audio is just bits it's
not text yet it's just bits so then
we're going to try so again remember in
the python world try is like try to do
something and if it doesn't work for
some reason do something else uh again
when we're doing API calls when you're
interacting with the operating system
you want to use try a lot so we're going
to try to have phrase so phrase that
variable we created is going to equal r.
recognize Google that audio that's
coming in so the audio that came in from
the microphone right we're going to try
to recognize the text and then we're
going to set it all to lowercase so that
lower is a method it's going to turn all
the text to lowercase why do we turn all
the text to lowercase because we have to
parse the text right we have to read the
text uppercase letters are different
than lowercase letters especially in the
Linux world so if we have a mix of
uppercase and lower cases or the
computer thinks something is a name so
they make it an uppercase or whatever
else when we try to parse we might run
into some weird issues so by changing
the capitalization scheme uh to one
standard for all the characters in the
text it makes our life easier to parse
right so the audio that's going to come
in we're going to turn it into text and
we're going to make it all lowercase and
then we're going to print that out on
the
screen except accept uh unknown value er
error basically it's going to say I do
not understand accept request error as e
uh could not request results from Google
speech recognition service and it
basically tells you why right so if
there's some kind of weird error it's
going to say I do not understand if
there's a request error then it's going
to give you whatever the request error
is then we're going to come down here to
something that looks very simple isn't
this simple so easy so first we want the
trigger word we want a trigger world
right CU remember this is simply on and
the computer is simply listening it
doesn't know who's talking again one of
the problems we get into when we start
talking about artificial intelligence is
that it doesn't really exist right when
people talk about AI they think about
like human AI or dog AI or cat AI oh
it's it's AI it's not as smart as a
human but it's as smart as a dog no no
no it's that's just just not how this
works right so anyways when this is
listening it is pulling in all of the
words around it so if you have 10 people
talking if you have the radio on or
whatever else it's going to be listening
to all of that and it's going to be
turning that into text so if you're
trying to trigger based off of words and
it's listening to all this crap it's
just never got to shut up right it's
going to be one of the problems so we
want a trigger word uh so this is
um oh in the Apple World hey s i i i
don't want to screw with your system
right that's that's a trigger phrase uh
that's a trigger phrase uh Google has
their their phrase Microsoft has their
phrase so the cool thing here is we can
put in whatever trigger we want and so
I'm just going to put in computer so
when I say computer so if I say
something if if I say something and it
hears and it hears computer in that that
phrase then it's going to start looking
for some other words if it doesn't hear
computer in the phrase it's not going to
look for any additional words and it'll
just Loop through right so basically
it'll only respond hopefully if I'm
talking to it if time and so when you
think about these triggers this is the
other thing you have to think about is
what word or words or phrase do you want
to trigger on right nothing magically
happens in the technology World somebody
has to code all this crap
so depending on how many different ways
you want to be able to say things like
time or whatever else you will actually
have to code that in so that the
computer knows what to look for right so
I'm just going to do this very simple
based off of single words so if time and
phrase so if I say computer and time
it's going to get the get it's going to
go to the get time function and the
response is going to equal the value
that comes back L if weather so computer
and weather is in phrase we're going to
go to the get weather function and we're
going to pull something back L if joke
is in phrase we're actually going to
send the phrase so when I say tell me a
joke about Lewis Rossman and bananas
that entire phrase gets sent to open AI
so that I can get that unique joke back
right so with get time I'm not sending
anything it is getting the value from
the operating system and it's responding
uh get weather we're doing some what are
called rest API calls and it's just
presenting back with this we're actually
sending our entire request to open AI so
that we can get that custom response
back from open AI then we're going to
have light so if I say computer light um
we're going to send again switch light
but we're going to send the phrase too
so basically we're going to S to switch
light here there's going to be an IFL
statement for if on or if off and so
that's going to be triggered so we're
going to send the entire phrase uh to to
that switch light function and then L if
I say dismissed so dismissed basically
says turn off break so break will break
out of the entire Loop else there's
something else we're just going to pass
at this point so pass just means keep on
going uh and then I'm going to print the
response here so whatever response I get
back from these functions I'm going to
print that response out on the screen so
I know I'm getting the response from the
function even if the computer isn't
talking to me yet then we're going to go
down here to gtts Google text to talk
service so we're going to say try and
this is kind of like the standard so my
OB OB equals gtts text equals
response Lang equals that language that
English that we said above and slow
equals false my ob. saave response. mpp3
this is going to save into my default
root directory so it's GNA save just
into the Eli directory and it's going to
be saved as response. MP3 so again
depending on what you're building you
may want this to be a dynamic name you
get to design that we're just going to
use a static then we're going to do os.
system
afpl response. mpp3 so there's an apple
file Play application that's built into
apple and so I'm going to use this in
order to play the MP3 file and that's
what's going to get the computer
speaking to us uh so this is something
again uh depending on what system that
you're using uh Ubuntu uh it's like 321
MPG or something like that is the is the
go-to basically there's a whole bunch of
different command line MP3 players out
there all you need to do is install one
of them onto your system and basically
all we're doing is we're simply
triggering that MP3 application to run
and play this particular MP3 and that's
what's giv us us the voice and so this
is simply the while true Loop that goes
through and then after Loops response
goes back to nothing phrase goes back to
nothing now one of the things that I
will tell you um so I'm I'm here in
downtown Asheville and Asheville gets a
little bit loud gets a little bit noisy
with this listen on the microphone it
will listen until it stops hearing sound
or until it times out which is something
like 20 or 30 seconds if you are in a
loud environment this whole system may
come to a a very slow crawl basically
because it's going to keep listening
until it stops hearing anything so if
you have trucks in the background if you
have road noise if you're in an office
environment where lots of people are
talking then the sound level is going to
still be high enough that it's going to
keep going until a default Loops out of
it and so that's one issue that you
might run into so if you're going to be
playing with this try to play with this
in a quiet environment and there's a
whole bunch of stuff with how to deal
with microphones and ambient noise and
all that kind of thing we'll play with
that at a later dat uh that's that's one
of the troubleshooting so if you're if
you're using this and it like seems to
be working but it takes way too long and
then you realize the environment's
pretty loud that's the thing is
basically it's listen listening to your
environment it's waiting for quiet and
if your environment is never quiet then
you might run into some quirks again
especially if you have like loud and
alloud AAC system you might run into
some problems with that anyways let me
then close uh this while true or Shrink
that down and we can go up and we can
see the different functions so the get
time so if I say computer what time is
it all it's looking for is computer and
time it's going to go to the get time
function current current time equals
date time now message equals string of
current time uh string of f time and
what this does is oh percentage sign a
gives you the day percentage time H
gives you percentage sign H gives you uh
the hour colon is just a colon and
percentage sign percentage uh sign M
gives you the minutes so it'll tell you
the day the hour and the minute
that is going to be turned into a string
so I don't want it as number or
something else because it has to get
spoken that's value is going to go to
message and then we're going to return
message so Friday at 2:15 is going to be
the value of message as a string that
will then get returned and we go down to
the while true Loop right so if time in
phrase so that message is then going to
go to resp response and then down here
response is going to get turned into an
audio file and then the time is going to
be spoken to you that's basically how
that works and how this function works
right so this is all very simple
basically you just get the time you
format the time as you want it you don't
want a lot of additional stuff you turn
it into a string and then you return it
that's how you get the time then we go
down here to weather um so with this at
the end of the day we're going to be
using open weather so open weather uh
has an API that's very easy to use um
the issue is is that we need the
latitude and the
longitude so how are we going to get the
latitude and longitude again one of the
things you have to be thinking about as
a technology professional is you
actually got to get this data from
somewhere nothing magically happens
right so basically in order to get the
latitude longitude so if we're using
open weather we need the latitude and we
need the longitude so how are we going
to get that so the first thing that
we're going to do is we're going to use
IP fi. org's API this will get us our
external IP address right and so our
external IP address is for the premises
that we're at right there's an IP
address that actually connects us to the
to the internet and we want what that
external IP address is so that's all
this call is going to get us and so here
we're going to get our IP address and
it's going to get turned into text so
text of IP address is IP address then
ipen ai.com what this does is this gives
you a whole bunch of information that's
stored in a database about your external
IP address your latitude your longitude
uh the city that you're in the state
that you're in uh the provider that you
have there's about 20 different things
that it basically stores in that
database so for IP data we're going to
do request.get
ipy ai.com
Json and then we're going to plug in
that IP address that we got and we're
going to bring this back as a Json value
right so once we then bring it back as a
Json value we can come down here and so
now weather is going to equal
request.get API openweathermap.org data
weather blah blah blah lat equals ipor
data right so this is a Json value ipor
data lat so that'll be my latitude ipor
data lawn that would be my longitude and
then we also need our API key here uh
one of the things if you're going to be
using the open Weather API is you also
should add in units equals Imperial or
whatever your particular units are um
the default units that it uses is
Kelvin why I don't know
if you just use their if you just use
their stock standard you just copy and
paste their API call with a lat lawn
you're going to get numbers that just do
not make a damn bit of sense and the
reason is is because you're dealing with
Kelvin I think it is so anyway you go
units equals Imperial you're going to
feed at the API key and then uh we're
going to turn this into Json so we get
this response back this will be Json so
message is going to then equal right
weather so weather that we brought back
and then Main and then temp right so
these are for in the Json the main and
then temp that is going to get us what
our current temperature is we're going
to be using an F string so that we can
add degrees at the end again this is
going to be a message that we're going
to be sending so we going to say
80°
39° and then we're going to return that
overall message and so this is kind of
the step and something like a rest API
call First you're going to get the IP
address then you're going to get the
geographic information about the IP
address then with that lat long from
that geographic information you're then
going to get the weather from there
you're getting a full Json response so
then you need to figure out where the
current temperature is and just pull out
that specific information so anyways
that's what the git weather uh looks
like then we go down here uh to the git
joke now you'll notice with these uh
functions up here get weather and get
time um the the parenthesis it's blank
now with get joke we actually have
phrase in here right so with the joke we
want a custom joke we don't want the
same old joke at the time we want the
custom joke so in order to get a custom
joke we have to send that whole phrase
to open AI so that we can get a custom
response so with this get joke all of
this is the same standard open AI stuff
I've been teaching quite a bit basically
you can copy and paste this from the
open AI API uh service or whatever we're
using GPT 3.5 turbo model because it's
Dirt Cheap roll system you're going to
be a comedian role assistant do remember
to limit this remember it might tell you
a really long joke that a short bad joke
is still funny a really really long bad
joke is just horrible right so anyways
for assistant we're going to say answer
in less than 25 words and then for role
user we're simply going to pass the
entire phrase right so we're going to
whatever that thing is tell me a joke
about you know Lewis Rossman and Bill
Clinton that entire phrase is going to
be sent to open AI open AI is gonna say
I'm a comedian this should be answered
in less than 25
words message is then going to equal
response choices zero message content so
you're going to get a Json response back
and this this is where what you can care
about actually is contents and then
we're going to return that message right
that's how that works uh if we then go
down to the light switch so basically
the light switch again we have phrase
because we're going to send the entire
phrase because once we're in the light
swich function we still want to parse
based off of or we still want to trigger
based off of what's in that phrase right
so if we go down here so if
computer is in if computer is in phrase
do one of these if light is in phrase do
the switch light send the entire
phrase and then here what we're going to
do is we're going to say if on in phrase
so turn computer computer turn light on
computer light on so on so we're going
to do is going to try serial. write U on
next line in message equals light is
on except pass so for some reason this
doesn't work uh we're just going to pass
so if the serial right doesn't work
we're going to pass so if on we're going
to try to serial write on to the
Arduino uh then L if off in phrase try
serial. WR off message equals light is
off except pass else message equals I
don't understand what you want to do
with the like so if I say
computer turn light blue that's not on
that's not off so this is the message
response that you would get back and
that is the end so these are the
different functions here beyond that
though is with this again we're doing
this whole serial right thing so up here
we're creating the serial connection
right so we have uh the serial module
here then we're creating the serial
connection using Dev USB modem at 9 600
bot again we might have to do a Google
search to figure out exactly what you
need to put in there and then we're
flushing but then we're doing the the
serial from
there what we then look at is this is
the actual code on the Arduino right so
I have my Arduino down here and this is
the code that is running on the Arduino
might actually have to make that even
bigger right so with this so on the
Arduino we're using digital port 3
digital port 3 as the the the port to
turn on or turn off the uh the relay so
the relay has wires that come to the
Arduino and this is what triggers it so
toine switch as digital port three we
have string is command we have the void
setup so this is where we set up the
environment serial do begin at 9600 so
remember in that in that serial
communication that we set up in p python
at 9600 this is where we're setting up
the communication here on the Arduino
Arduino then we're going to do
serial.print line type command on equals
on off equals off pin mode switch so
digital pin 3 is switch set it to Output
so it sends information out then we're
going to delay for 1,000 milliseconds or
1 second then we're going to come down
here to the loop right so in the Arduino
world this is not a computer it's a
microcontroller just continuously Loops
you tell it to do something it
continuously Loops forever and ever and
ever um if serial. available uh command
equals serial. ring read string until
that's slash n read string until next
line and then command. trim so the
command is going to be everything until
the next line and then we're going to
trim off any white space because
remember we always got to Tri trim off
white space with these kind of things
things so that we don't run into weird
problems right so here right serial.
write on sln so read until slash in we
down here seral WR off slash in so
that's next line so basically to read
until that's slash
n uh then it'll trim off any wh space so
on this is comment so for on if command.
equals on on digital right switch high
so turn digital pen 3 on if command.
equals off digital right switch low
which basically means off so with this
this is working I can literally just go
to serial monitor here so I go to serial
monitor
oops board
at let me get the board right so go to
tools you might actually have to do this
um
not programmer Port we go to Port we go
there okay now we do serial
monitor all right so on the serial
monitor
here um it's set and all this so we need
to set it down to 9600 so I then set it
to
9600 let me just refresh my Arduino turn
it off and
on okay and now from here what I can do
is I can simply type on
on see I typed on and it goes on I type
off and it goes off right so from here
with this serial monitor I can type in
the commands in order to control this
relay all we're doing with python is
we're having python send those serial
commands um to the Arduino instead of us
doing this manually so there you go we
buil our own little void voice
controlled python application this
allows us to interact with the operating
system it allows us to interact with
rest apis it allows us to interact with
artificial intelligence and it allows us
to be able to turn things on or off so
what I really wanted to show you uh with
this project today is how relatively
easy this type of thing is I know when
you start getting the technology world
and people start talking about
artificial intelligence and speech
recognition and iot it's like oh my God
how am I ever gonna do it well the way
you're gonna do it is because it's not
nearly as hard as you think it is right
again coming up with that speech
recognition package the person who
developed speech recognition that person
is really really really smart me
installing a package and then calling a
module and getting text back yeah you
know n so much again open AI people I
will never denigrate how smart the open
AI people
are that's not me I'm writing five lines
of code I'm making an API call I'm
getting text back and I'm doing
something with a text right uh this
stuff is a lot easier than people think
AI is hard apis are not right and so uh
and so this is a lot of the cool stuff
uh that you can do and again uh more of
the projects that we're going to be
doing this year again things like
Machine Vision right uh you can have
either as yours uh computer vision
services or other computer vision
services you can literally send a video
feed or an image and it will just simply
send you text back so if you if you know
how to plug in the code in order to make
the API call you can be getting that
text back and then again you're
triggering based off of relatively
simple text it's not nearly as
complicated uh as you you think it is uh
so yeah so that's the particular project
uh so for this again if you're going to
be doing this uh the warning that I will
give you is pi audio will either be
really simple for you or real pain in
the butt can't tell you wish I don't
know your operating system right if Pi
audio goes well for you thank your lucky
stars and keep going if not you might
have some troubleshooting to do that's
just how the world is uh the other
troubleshooting thing to be thinking
about with this is again if you're in a
loud environment uh that might cause you
some problems uh simply because when you
trigger the microphone the microphone is
going to keep listening until this the
the noise level gets below a certain
threshold so if you have trucks if you
have traffic if you have loud people if
you allow HVAC
system this whole thing might be quirky
as hell because instead of you
talking stopping it hears a silence and
then it triggers it'll just keep going
essentially until it times out and
that's going to be an entirely different
experience for you so you know that's
just kind of how it is you know find a
quiet environment that type of deal um
beyond that again these are basic
function calls uh these are basic um you
know if if El statements you got a w
while true Loop in there all of that is
relatively simple uh with this if you
want to do the open AI uh you do need
the open API key uh so you have to go to
open Ai and in order to get that API key
uh you give them their credit card
information and basically do make sure
to go in there there's a a limiting
function so with open AI you can limit
how much it's going to cost you so for
me I actually set it at $10 a month and
I never go over $10 a month so that's
something you may want to say you you'll
need to get the API key in order this
project and you may want to set that
limit uh the other thing is that open
weather for open weather uh there's a
free service up to a lot of requests um
but you also do need the API key from
that uh so if you want to be able to use
use that go get your your your free open
API key or your free open Weather API
key in order to be able to get that
weather information uh but with the rest
of this the time and all of that kind of
stuff that should work for you uh as far
as the Arduino goes uh this is basically
just an Arduino Uno so this is a four
five-year-old Arduino Uno that's all you
need uh we just have a single uh digital
line uh that's connected to a relay uh
so I like this particular relay it cost
you about $30 off of Amazon but it's
nice because it's actually a nice
plastic box I don't have to worry about
electrocuting myself or anything like
that um but basically the Arduino is
then plugged into the the relay the
relay is then plugged into the light and
that is what makes that whole system
work and yeah that's about all I got for
you so uh so let's leave this on a joke
let's leave this on a joke here we go
computer tell me a joke about artificial
intelligence and light
bulbs why did the artificial
intelligence researcher buy a bunch of
of light
bulbs because they wanted to shed some
bites on bright
ideas and as always I enjoyed uh doing
this particular class and I look forward
to seeing the next one