what's up what's up what's up what's up
welcome back everybody uh today we are
working on our video editor so pitch for
the editor is uh I produce 2our long
vods approximately daily um there's
maybe an argument to be made that
shorter videos are a better format for
YouTube right when I watch other
people's stream other streamers videos I
see a lot of like 20 minute stuff and I
do notice that I Trend to if there is a
20-minute version of the thing I want to
watch or the 2hour vaud I'll probably
click on the 20-minute one there's a
reason for that the little monkeys in my
brain they like it a little better so I
want to experiment and see if uh cutting
this type of content down uh can produce
something that's interesting and like
worth watching um but my initial
attempts were uh I struggled like just
popping open like kdn live which is
maybe I don't know maybe there's a
better option but it's just like a video
editor on Linux that I wanted to try so
I
yeah I tried cutting it down and I was
like oh this is going to take like at
least 4 hours to cut down a 2hour video
that's no good the point of me doing
this at all is supposed to be that I
find it fun I don't like you know if I'm
going to make videos and put them on
YouTube like I'm trying to just code and
see what comes out of it and so like
putting in like way more effort on like
the video editing side versus the coding
side it doesn't make sense it doesn't
make sense um and so the question is
well can we make this easier in some way
and so the idea here is that it would be
kind of fun if we could uh autogenerate
like a script for the video and attach
that script to the video editor itself
so my thought is that I could like
select sections of text of things that I
said and said I want that in and I don't
want that in and maybe like you can like
start making like a tree of tags and
stuff so that you can like try muting
stuff and unmuting stuff and your
playback will automatically hop around
um and show what's selected uh and then
like I want to like be able to export
that pretty quickly so hopefully avoid
all sorts of like re-encoding stuff cuz
the vast majority of the video that
you're exporting should be a copy paste
you might have to re-encode some frames
that don't align with like if frame
boundaries but
whatever um so yeah I drew a little
mockup here this is what what I'm
thinking this is me in the corner and
here's like my little to-do on the top
right and here's my little code and
there's like a the timeline in a script
um and today's topic is audio so we want
to yeah I guess audio is important do I
don't really need to like I don't need
to justify in any way uh if I'm cutting
down my stream I need to hear it So the
plan for today is you know really basic
play some sound get some sound to play
and then play that sound and we'll see
what happens we'll see what
happens uh so I guess uh we should show
current
state so current state is this thing um
we have a open window that's opened with
glfw he gives us an open G context we
take the video from ffmpeg we get some
frames we shove those frames into an
open G texture and then we display them
there was a little bit of pain with like
color conversion stuff but not too bad
yesterday yesterday we looked at uh
aspect ratio adjustment so this is oh
this feels so good this is so much
better than it was yesterday um anyways
okay let's work on audio
so uh what's Li for the goey uh no
lib yet I'm trying to avoid making that
decision as late as possible because I
don't want to think about it so uh we'll
get there eventually but for now it's
just openg GL
uh I'm talking fast yeah I should I
should slow down I watch all of the
videos that I watch at like 2 to 4X
speed and it's really just like
broken my brain I just sit
there that's what's happening in my head
and I got to I got to slow it
down
um okay what am I looking at uh okay so
if you look up like audio library
C uh there are a few examples I looked
at uh I clicked on this one this one and
this one uh this one stands out because
it's the first on the list this one
stands out because it's made by Mr Zig
himself uh and this one stands out
because it's a single header C library
and if I just sort by GitHub Stars I
think this guy wins and because I don't
really know anything and I like the idea
of a single- header Library I'm going to
use this one um I didn't really you know
there's not really much thought going
into this it's just like it doesn't
really matter right like in reality all
three of these things are probably just
rapping alsoa and in reality if I were
like if I wanted to try a little bit
harder you know it's probably not that
hard to just like pull out you know to
like build on top of ala alsoa is
already a library it's just like but
then you have this this allows me to
like think one layer higher like now I
don't have to think about like ala
versus pulse audio versus whatever the
Windows does versus whatever the
Max Mac OS does like this is just
going to give me something there's also
sdl sound yeah I did look at that too
but it's like if we're going to pull in
sdl then we might as well run to the
video player with sdl and then it's like
sdl's like a lot more than what I want
where these guys are kind of more
scoped uh yeah so let's yoink this guy
and I guess we'll just shove him
somewhere maybe in source is where he
belongs and maybe our build script we'll
have to uh compile him in some way so we
should do like add a c source file who
is uh Source mini
audio and we have to Define like the
implementation flag here I wonder
usually I split out like a a full cile
for this but I wonder if you can just
get away with
uh like defining stuff here you
could add like A-D flag to this Flags
thing but it would be nice if there's
like I I don't think that you're
supposed to do it that way um I think
there's like add
[Music]
C okay whatever we'll do it the other
way we will make a source mini audio
imple C and we'll look at mini audio. oh
my God ow there's so much code in here
and then we'll just do the like slap
this guy over here boom boom
boom and
I think that's it I think that's it in
our build script we need to add this
guy to our compiled files that we're
using and then in our main file we
should be able to or in our C Imports we
should be able to see
include mini audio. and I guess also for
all of this to work our build script has
to say where to find that header file
which I don't think we're doing right
now uh oh we are we that's already there
okay so I think we're ching
this should
compile and if it doesn't compile then
we'll fix it but it should
compile give him a sec he's
thinking the guys in the in the wheels
they got to turn the gears you know
there's a lot going on under
there okay so now we have our audio
library and the question now becomes how
do we want to test this the first thing
we want to do is like make something
that plays some sound and it's probably
easier to make like a new playground for
for
this probably I don't think cuz like I
don't think there's anything in we don't
have like any infrastructure in our
existing app that has anything to do
with this so it's probably easier to
just start from
scratch
uh so maybe we what we
import s and import and then we also
import our like C
bindings and then I guess we don't even
really necessarily need this allocator
yet we don't really know if we're going
to need if if all the allocation is
handled by mini audio then we don't
really have to do anything ourselves you
did some refactoring off stream just a
little bit I just took out like the
player State um so like just the stuff
in the main Loop where we were saying
like uh can we pause that just kind of
got shoved into its own class oh I guess
I also fixed uh a little bit of the like
keep allocation stuff that we needed to
do to make the callbacks work um but
nothing too major nothing too major
it's all basically the
same okay what's in this guy he's going
to tell us what to do somewhere uh so
this guy has like a lowle and high level
API I was kind of skimming the API docs
before stream and I think the lowle API
is fine like one of them is like hey I
want to like take audio files and play
them at given times and like mix them
together and stuff and what the lowlevel
ones just like hey I have like an audio
device let me like throw some some like
audio frames at you and you'll play them
and I think that this probably matches
what what we want from FFM Peg a little
better FFM Peg is going to give us uh
audio samples and we're going to have to
like dynamically inject them as we load
them uh especially if we start like
seeking around then it's not going to be
like a linear thing you could argue that
you could like extract the audio and
like upload it to mini audio system once
and then just have it play but then you
have to really worry about like desyncs
between the audio and the video so just
like shoving frames in as we get them I
think probably makes more sense and if
you look at this like lowle API it's
pretty straightforward right they're
like hey I want a device here is the
configuration for that device please
initialize it and then like start
playing sound on it and then all we have
to do there is like give it some
function that it's going to call Hope
with no user data what up what these
like C libraries and not oh never mindo
scared me okay there is user data here
we must just have to like extract it
from the device or something uh so we're
going to have to like set up some sort
of like system to be like I have audio
over here I would like to hand it over
here to your call back and then when it
runs it'll like ask for
it not really sure about like the
threading model here yet but we'll
figure that out pretty
quick so I guess we just copy paste the
sample
and start going from there so this has
to be converted to a zig
function so he's going to take in a
device which is
uh whatever the this thing is a ma
device
pointer and we'll call this dude what up
with people and naming variables after
their types this we did this at like at
one of the companies I worked at where
everything they would they would name
like bu B do stuff and some poter like
bull star would be like PB do stuff but
we only did it for pointers and booleans
I don't know why pointers and booleans
are special
right and these guys are doing the same
thing where they have pointer P ma
device P output P input but then frame
count like why shouldn't this be like
u32 frame count if we're going to like
be shoving our types into names is that
hungaryan notation I think
so like why why why why do it at all I
don't understand see like this makes
more sense to me right if you're going
to do it shove your L for long like like
you know s su's name like if you're
going to do it just go all the way I
don't understand this like half ass
version you
know hungara notation more like Windows
notation oh yeah they do do that a lot
huh but they also yell at at you I've
seen like those like uh Windows types
are always in
uppercase uh anyways not that it matters
it's just one of those things where it's
like I don't understand it uh this was
what was output was a void
star which is not I think it's uh is it
void here in the I can't remember if
we're supposed to do star void or star
anyo take
like I think we're supposed to do this I
think that Zig says uh void pointers
aren't allowed I
think I
think uh oops forgot to call in
[Music]
here and oh this is a pointer to
a
const uh right const void is not a
constant pointer but it's a constant
element yeah you can tell that I'm a
little bit Rusty when I'm not sure which
when I'm not sure where the uh where the
cons
goes uh this is a c u or I guess it's a
u32 um I guess this is a c ma unit
through to I guess we should be true to
the
input okay and then this is going to use
the SE calling convention so we have to
say hey Zig don't do whatever your doing
this better just do what C
expects and then eventually we'll have
to do
something uh but for now let's just
start by converting their little
example into r or not rust what language
are we working in again Zig I know
what's going
on uh so we'll start by getting some
config and this is going to come from
this guy
and I guess
maybe just as we get going it's nice to
stay in like a compiling state so let's
get this guy compiling at least a little
bit and then we'll continue
on do I use a 4k monitor nope little
1080p
guy
um but which is fine
because
uh 144 FPS is sick when I upgraded to a
144 Herz monitor I was like holy crap
this like completely changed my life
like for games it's like it's like no
wonder I couldn't name before uh but I
really doubt that I can drive anything
above 1080p at 144 FPS anyways so it's
like I don't know and plus like I my
eyes are so bad my prescription's like
what like Min - 11 or some and so
like I don't really know if I even like
notice everything's so blurry
anyways
uh minus 11 yeah dude it's bad look at
check out like the thickness on these
bad boys I don't know if you can see it
cuz I can't see my monitor when I take
them
off uh was I doing I was going to try to
compile
this are you going to be using some
library to handle video codex yeah we're
using FFM Peg we're using FFM
Peg it is fun though I don't know like
uh I've I've done a little bit of
working with uh the h264
spec it's kind of fun it's kind of fun I
don't think I've ever written a decoder
straight from scratch but there's like
I've had to like look at I don't
remember what I was doing something
about packetization made me look at it a
little more detailed but like that just
crazy and would be a fun project
actually writing an h264 decoder that's
a good stream idea actually I should do
that at some point because there's a lot
in there like cuz they do like they do
this weird like frequency domain
compression if I remember
correctly where they like for8 transfer
transform the stuff and then crop out
anything that's high frequency cuz like
who cares you're not going to
see it anyways and there's like that's
why you there's also something about
like the
compressions there's something about
like when you see like dark rooms
on uh like your low bit rate streams
like the darks are so impossible to see
and it gets like really blocking and
pixelated there's like some interesting
reason behind that that I can't remember
right now but it's a good idea it's a
good
idea I haven't either but I he there's
h265 now yeah h265 I don't think is
really it doesn't really benefit much if
I remember correctly we used h265 for
security cameras at the company I was
working with because it supported like
sub streams or something so if you were
looking only at one area of the stream
you could take an H2 h265 frame and only
send like one tile of it which was
really nice but I think that like
outside of that I don't remember
thinking that it was any
better uh what was I doing I was going
to build our new application so I guess
all of these flags that we're passing to
our video editor executable are going to
be used on both so I guess we do
something like
uh function make
exe who is going to take in our build
and he's going to return maybe nothing
to be honest he might return nothing and
I guess we'll just take in the
name and the
path and I guess we also need
the can you extract these over and over
again
these things probably I can't I I
imagine that's fine I imagine but we'll
see so the name goes now is just a name
and the path is
path and then all we have to do is call
make exe with video
editor and Source main Zig and do it
again with uh audio test and source
audio test there we
go expected three arguments found two
but these guys need Little Bees in front
of
them little bees not big bees little
ones signal processing is like 95% for
analysis that makes sense uh I can't
declare options twice okay so I need to
do this Target and option processing
here and I guess what I'll do instead is
cuz I'm feeling a little lazy is we'll
stick them in static name space or I
don't know what it's called in this
language I don't know what it's called
Global so this is
a stood build resolved
Target and this is a stood
build uh built-in optimization mode
nice and then here we can just click
these consts boom
we good is he chilling we're close uh
Undeclared div identify ma device so
let's clean up some of these guys so
this is in the C Imports so we shove a
little C dot on
it uh dependency Loop
detected Pub const ma data
device proc
oh no that's not good C import Zig 6153
dependency Loop
detected
uh so does this mean that we just can't
use this library from Zig so a very real
possibility uh okay so what is he
complaining about he's
saying what is the dependency Loop
I actually don't see
it I guess maybe ma
device might
have I don't
know I don't know but this might be an
indication that do you have to like
Define the impo pre-compile bar like in
the C file I did no no no I shouldn't do
that in the zig Imports
um that's
annoying that's
annoying
so I wonder if they're mad because ma
device has a ma device data proc in it
and
this uses an MA device in here like I
wonder if I do
this if it likes that better of
curiosity just out of
curiosity uh that did get me into a
different
error uh that's annoying zigs C import
dependency Loop
I build success by modify this guy is
not a native English speaker for
sure this guy just
commented just like oh he's just posting
another
error expect Behavior build and run okay
it's pretty funny uh yeah interesting so
this
I
imagine okay so we can maybe write like
a little sample like a little test case
for this uh okay let's
uh what about erasing the type and just
using any a I think it's a problem with
like the import of the library itself
though so if we
um let's make a is there test do I have
like a test Zig project anywhere
no I suspect that it's um related to
if I were to make a test H and in here I
had like
struck
my thing and a member of my thing is
like
my function call
back and if I declare
like my function
callback as a pointer to a function that
takes in a pointer to my thing
uh how do you do that in this language
my thing star
thing I think this is what he doesn't
like I think so we could test this by
doing uh Zig
trans Zig
help translate C translate C test out
H unknown type my function call back
uh
oopsies uh and then do I have to do this
up here or something and then do a
struct my thing as a forward declarer is
that how I have to do
this I guess we should test this in C
first because I don't remember my syntax
well enough
uh unknown type name my
thing oh so I WR struct
here unknown type my function call back
oh cuz I named did the wrong
thing okay uh so if I oh no he's fine
with
that he is fine with
that uh
okay so what did
I what do we change here it was this was
the state it was in
before
where this thing yeah this guy was in
device so I don't understand why
translates see the whole mini A.H that's
not a bad
idea let's see what that does ZIG
translate C mini
audio. uh and then we have to say like L
be
thread oh that works too
okay interesting and then in here do we
see like
uh
ma I think it's something with the build
script I don't know what yeah okay so
let's let's uh let's get a little bit
crazier in
here
uh let's see let's see let's
see so let's try if you think it's
something in the build script we can
check that by doing
um in our audio test we can import
instead of doing this import c. Zig we
can do c import
directly and do a c include of mini
audio. so now build script is going to
we can take build script completely out
of the question by doing Zig build or
sorry Zig
run source audio test and he's going to
say hey where the is oh he's going
to say where's mini audio so I can say
uh include
Source okay and then he's going to say
uh no P thread so we'll type LP thread
and then dependency Loop detected is
still there
okay so the question is is why does
translate C do anything different than
this um
and it would be interesting to try to
repr this now with our little test case
here so if I'm correct and it's about
this like this back and forth
resolution translate C didn't Define
many audio implantation right so it only
succeeded on subset but we're not
defining that in our thing
either in RC import we're not we're not
defining it so it should be the
same um okay so let's
try if we were to
say con C is equal to C import
here and in here we were try going to
try to see include our little uh test
file which is test.
H
uh then we did Zig
run test
Zig and include dot uh he needs this to
be
public no issues there
there what if we try to use this thing
now uh what's in test.
h
uh const value is equal to my
thing is a c my thing equal
undefined and then here we can just
maybe like stood debug print V so it
doesn't get compiled out
okay
um and then here we're going to import
uh standard
Library
okay okay so clearly not as simple as
this case like I thought um it would be
interesting to try to break down the
problem uh to the point where we could
fix it
that would just be that would be kind of
fun so let's pivot we'll pivot no more
audio I want to know why I can't include
this file we let ourselves Rabbit Hole
here cuz it's the fun thing to do the
other option would be we could just use
a different audio library which is
probably like maybe more sane but uh
shut the
up you can't make me do anything this is
my
stream
okay so let's do test. zig and try
importing mini audio. instead and let's
just see if this reproduces the
problem which
doesn't why is it different now oh
because we actually tried to use
something
probably uh okay wait wait wait okay
let's go back let's go
back um here and let's go to audio test
and start commenting stuff out
right if
we do
this
and
this I'm actually going to get it to
compile by typ racing the fun function
pointers uh yeah so like uh
in uh sorry brain's melting in the mini
audio. you just type you just replace
all P
devices with
uh pointers to any
opaque why where is it okay
so yes this is fine and then as soon as
I try to use it probably it starts to
break yeah could also do it in the H but
how did you do it like you're so in
C you did it in C import. Zig so you
modded the
compiler
oh I didn't oh I see okay so when I call
C import he calls translate
C he calls translate C on the header and
so when we call translate C it was
working fine probably and then he
generates this like translate C file so
if I went to sort or this like cimport
file so if I did a zig
translate C mini
audio. LP thread and then I exported
this to mini audio Zig just as a
test and then in audio in our like audio
test thing instead of calling C import
here we said const c is equal
to import mini audio zig
and then we tried to run this I imagine
that we'll get the same
problem yes okay and then you're saying
you mod C import. Zig to type erase the
device pointers so if I go into mini
audio. Zig I should be able to go to
uh ma
device in these function pointers
here and just change these into
uh any opaque so I should just search
for uh like
this no this one doesn't matter it's
maybe only these
ones uh ma device notification proc so
this guy again we just need
[Music]
to cons any
opaque oh okay so it is only a few of
them okay interesting so it would be I
I'm also kind of curious about like what
the dependency cycle even is here so
it's something related to these guys
uh so I'm curious I'm just I'm now I'm
just I'm just curious
here uh is there any like what's is
there a very simple version of
this it's interesting because it's not
oh it is maybe all of them so if I do
this one and put this guy back do I get
an error
here yeah okay okay so ma stop proc
references ma device
ma
device must have an MA stop proc in him
somewhere okay let's just out of
curiosity what if I did
this the dependency Loop is still
detected so it' be I wonder if we can
like can we almost
I just okay what if we just get rid of
everything except
for these two lines
right like let's just get crazy here and
see if this is enough to reproduce the
issue
uh
oh uh now the
uh we need to at least do something here
so cons
device VAR device is this thing equals
undefined uh unused local variable so uh
this needs to be const and this goes to
device no okay so that's not
enough interesting so what I wonder
where this is coming in I wonder if we
can we look at
like uh
loop
protected so where does like this come
from so he
is going through some form of dependency
graph
and I guess he's probably
finding I wonder if we can
like do I want to build the compiler
right now is that what I want to
do feels like too much work but then
again just randomly probing is also a
lot of work so
like what are we doing
here let's undo all of these changes
that we just made I want to try to like
start bisecting so if we can at least
find a way to
start stripping this problem down to a
point
where it's simpler that would just be
very useful maybe for our own mental
model but maybe also for adjusting that
bug
report okay so let's go into ma device
and just start deleting things out of
here and seeing if that causes problems
cuz like I think that if we're not using
this
stuff maybe it doesn't cause a
problem uh where is my sorry not this
stuff where's my struct ma
device
here CU I don't know if like any of the
if the way we're compiling it right now
none of these like member variables
should be used anywhere I don't think
I could be wrong but let's just like see
what happens if I do
this uh well and then let's also go to
this ma. proc and send it
uh we'll put this back to an opaque
pointer just to see if we can get past
these compilation
issues uh so this is
any any
opaque okay sck so that compiles so we
can just start like nuking code
until until it starts compiling cuz we
know that we were in a state before
where we were compiling so let's
delete as much as we can
here uh okay let's go back where was
that
stop proc this is at line 5 553 we need
to remember that
553 so this is above it so we should
theoretically if I just delete
everything below here I don't I think we
should still
compile uh oh but he's going to get mad
about unused identifiers which is a
little
Annoying a little annoying but it's fine
so if
I just delete all this
stuff oh wait it says that un unused
identifier ma stop proc didn't I leave
this in here am I
high okay delete all this oh it's
because I when I delet it up here I
probably nuked it by
mistake yeah it's here so
let's right like if we do this delete
everything and go back I think we were
already in this state before and it
didn't cause a problem
root struct toile mini audio ma device
config in it oh okay so we need to keep
ma device config in it as
well there's also Zig reduced but I've
never used
it Zig
reduce what is it doing what is it
reducing
Zig reduce DH that's also just doing
something what the minimize a bug
report oh okay did that just like
up all sorts of stuff in my directory no
we're chilling it's compiling
itself oh interesting
interesting uh okay what's like the
easiest way to bisect this cuz I know
there is a way we just have to figure it
out like if I I guess we just start like
if I delete all this obviously that's
going to cause
problems obviously but how much can we
delete uh what a pain what a pain what a
pain what a
pain proper Sur culture gets suggest the
help flag says blind no it's cuz
when I ran it with help it did nothing I
already I tried that look when when I
type in D help it's like starts doing
something and I was like I don't know
what like the doing something is that's
scary
okay okay
okay so clearly just deleting
everything bad solution
here
but oh
is there an easier like another way we
can
bisect it's interesting that like it
it's clearly related to something else
right cuz
like my ma
device oh it must be someone who's using
ma
device hold on hold on that seems
likely so if we just delete all this
stuff
delete this delete
this still dependency Loop
detected like how much can we delete you
know Ma device config in
it so this is what we're using
here who's using ma device
type I guess we already saw that it
doesn't depend on what function we're
calling just that we're importing this
at
all oh what a pain what a pain what a
pain what a
pain what if we just
delete oh wait wait I know how we can we
can let's leave all this in for a second
we're going to leave all that in and
we're going to go to our audio test and
can we change this to do something else
like
c ma device is equal to undefined if we
do this does the bug stay no okay so it
depends on us calling this
function okay so what if we just keep
this and delete everything else right
like let's just start going
crazy so device config in it requires ma
device
config and
oh well first what happens if we just
say this
returns void just temporarily I know I
know I know we shouldn't do that but I
just want
to we don't really care about like it
this being reasonable you
know okay so if he doesn't return the
thing then this stops breaking as well
which is
surprising so it's related probably to
the instantiation of m a device
config so if we just say Define right
here this is CMA device config equals
undefined I wonder if that still
triggers the dependency it does okay so
it's in the device config we must like
what if we like just zero out this
struct
oopsies oh what did I just
do okay what if I just start like
deleting stuff from
here right just zero them the out
no dependency Loop okay so one of these
things is causing the
loop data call
back notification call
back maybe well it could be a
combination of multiple
actually so let's see
um bisecting
bisecting ah okay wait wait I think we
found it I think we found it we found
it okay so if we have an MA device
config who uses an MA stop proc who
takes in an MA device and then we try to
initialize the ma
device oh we don't have to call in it we
were just using ma stop proc stop proc
has a ma device and Ma
device has an MA stop proc so what the
does the config have to do with
it if I just do this does that trigger
it no right so there's something about
like a reference to a
thing that was not very uh clean words
that I'm saying
there uh but it seems like
it's I think it's like
struct
uh so we have uh ma device config with
ma
stop proc we have ma
stop proc references ma
device and we have ma device references
ma stop prod and instantiating this guy
causes the
problem which means that we can
probably reproduce this we can probably
like take only this stuff
out delete everything else and then copy
paste it back
in So this goes here and
then uh struct ma device this stuff
comes back in
okay delete everything else take this
stuff and shove it back in
here no does it depend on the order did
we up the order by changing
this uh okay so I guess we'll just write
down the lines that we pulled this
from
so this is at
uh
728 or maybe there's more to it maybe
there's
more this is
552 and this is
at oops we had a
ma wait where does ma device fig
go uh okay hold on hold on I think I
something up we're we're
restarting we're going from
scratch why is this so interesting yeah
I mean I'm really not really saying or
doing anything so there is it is a
little odd
huh okay Ma
device config
and this is at
line
752 and in here last time we oh I guess
we should delete first we delete first
so we go from here and we say we only
want I don't even remember
which thing was it was stop callback was
the thing that caused the bug if I
remember correctly
so here if we run oopsies
oh No undo
okay Zig run this we should see the bug
no because audio test. Zig doesn't exist
because we need to do it from
here okay dependency Loop detected still
there okay then we want to pull then we
we also modified ma
device and we basically deleted
everything in here except for on
stop still
broken okay so now the theory is that we
should be able to reconstruct this with
only the relevant players so this is at
657
um this is at 728 so it goes
below and then we had uh ma stop CR here
at
552 and so if we delete everything we
just put this stuff back
here does it cause the same problems yes
but what if
we so what if we put this down here
then yes okay okay wait we we've got it
we've got it we've got
it so we were in a state earlier though
where we copy pasted in some other order
and it was
fine but I don't know how we got
there maybe I just
misunderstood and then let's just make
sure that all of this stuff goes away if
I write any op here
it does
interesting okay so this is at
least this is at least a good test case
where we've really like stripped it
down is there more to say is there more
to I guess maybe we should we could try
also doing the same thing
um we can we can try taking
out this like mini audio referencing
so let's try to make another
Repro
um so what is this this is a struct
uses two different
structures use the same function call
back and that function call back
references one of those
structures so we should be able to do
something like
my
thing two references my function
callback
and if we then compile
this then
maybe maybe we see the
issue so C
import
test. and then try to construct a my
thing
to and then say discard
V so here if we try to Zig
run uh we have to go to the right
folder Zig run test
Zig Zig run- i. test
Zig dependency Loop detected let's go
baby let's
go okay let's rename these things to uh
something that's like a little bit less
like a little bit clearer so we'll call
this function call back we'll call this
us
a type
def struct
a struct b course this is a and this is
B and then here we try to import
B so is it related to translate C or not
um I think that translate C is
generating some Zig that is UN
usable so I think that here if we take
out like a uh if we try to instantiate a
here it's
fine yes when we try to a b here it's
not
fine cool and I like does it matter if I
put this like if I swap order of
something does that
matter no I don't know
okay
so we can go into this bug report now we
can say uh hit this while trying to
use mini
audio and we will maybe link it
here um well found
a stripped down
Repro which might be
helpful uh okay so our Repro
is
uh what how do you do this test
H let me copy paste this
stuff and test. Zig we do
this I guess we don't need this import s
anymore we just need
this uh let's just double check that we
don't want to submit an invalid thing
note uh note if we try to instantiate ca
instead of
CB then it is not a problem oopsies oh
crap it is not a problem okay what
version are you on yeah I'm get we'll
get there we'll get there we'll get
there um Zig 02 if but if you're trying
to nag me to put in the bucker Port then
uh don't worry we'll get there how do I
can I preview this yeah I can um
oh you want me to write Zig
here oh that's kind of
neat that's kind of
neat I think there are nightly builds of
Master available uh yeah I guess oh I
guess I should try it on Master because
if I don't try it on Master we can't be
we can't be sure that it hasn't been
fixed already
I'm using a newer version does not
work I guess uh not that I don't trust
you but I don't trust
you no offense you're just a guy on the
internet right I can't I can't trust
other
people uh where are the Linux
SES direct download downloads page uh
Linux is this Master yes Linux Master XA
664 Zig
Linux okay uh so we can say Z what was
our Command it was Zig run but then we
can just kind of slap a little this
thing in front of it
okay
yep cool
so uh observed on
Zig version
oopsies wrong
window Zig version
Z 12.0 and nightly
this
thing EIC pogan good eye good eye
uh run
with
[Music]
Zig
run and then boom
okay I'm not going to go as far as
trying to fix it although it would be
kind of fun to um oh I guess we should
also say uh
that uh
here we'll do this if we try to change
from C A to CB then it's not a problem
if we
replace um yeah yeah the workaround yeah
yeah if we
replace the function struct a star
with any
opaque uh oopsies Zig
run and let's run
this
um expected oh oopsies this is a
void
y if we
replace struct a
star with void star in the function call
back it is not a
problem okay
hopefully that's helpful to somebody
maybe me in the future probably not
though but I like to think that maybe I
will be interested enough to try fixing
this later but uh okay
so we at least understand the problem
enough that we can work around it I
think that that is fine absolutely fine
and I'm willing to just plow ahead now
and what we can probably do is we can
choose to either mod our function
callbacks with void
stars or we can choose to
generate
the like do the do a translate C run up
front and then replace it in
there um and it would be nice to
document the changes in some
way so maybe what we're going to do
let's we could maybe get
crazy we could generate a patch for the
header then save the patch and and then
apply it locally or
something before compiling or we could
just like comment in it
modded I don't know if I like either of
those
Solutions we uh okay here's what we'll
do here's what we'll do we'll store the
original Mini
audio. yeah yeah yeah yeah yeah yeah
okay okay here's what we'll do we'll
take mini
audio. we'll save this and and in our
build script we will manually run Zig
translate C and then apply a patch
manually and hopefully the zig translate
C call is
deterministic enough that we can get
away with that and that way we like we
will have like very Concrete in the code
what the have we done here uh
because otherwise you'll have like this
imported header and it look it's really
unclear what what is happened to it
do I like that better or we or could we
apply the
patch and include the modified source
file oh I like that better
actually I like that so
um in our Oh wrong window where
am
I video
editor
so build. zig
maybe what we do here
is I already forgot oh yeah we wanted to
we want to patch mini audio and then put
it in like some cache output
directory um and I know that the build
system supports like generated
files so we can do something
like okay we'll put this in mini audio
shove this in here then in the uh build
Zig documentation they have uh like
instructions on how to generate files
and use them in future
steps my my attention again but it might
be nice to have the diff changes in the
history yeah I guess you could just add
the header and then mod it and then
commit them the mods but I'd prefer to
have it
like there I don't know there's
something about including a modified
dependency I guess that's the whole
point of pulling the dependency in but
it seems likely to me that in the future
not that it matters right
because I'm the only person using this
and I'll remember but like you the the
the worry is that I will import I'll
like up update to a new version of mini
audio and the update process needs me to
apply those same mods again so having
them like codified is kind of nice
because then we can see what the changes
that we were doing are and like we don't
have to like go back through history to
find them um generating Zig Source codes
probably the
closest so in the build Zig they add an
anonymous import
uh it's not really the same we're not
trying to add an anonymous import we're
trying to add something to the include
path uh but I guess add output file Arc
is probably the thing that I'm looking
for yeah so this will give me a location
to write the new version of mini audio.
and then I can run a system tool to
generate
it cool okay so let's
um let's create our
mod and in here it's just that in our
function
pointers oh okay here's what we'll do
uh we can just put this here for
now and if we run our audio
test he should start failing
again oh except our audio test is not
using C
import now you'll fail and we can just
go through and fix these one by one so
this is ma data
proc which is this one
ma a device notification
proc uh was this one was was a bad one
as well where is the notification proc
oh it's
saying over
here is that
it yeah it's just the three all right
cool how' you get into programming video
processing is this something newer do
you have an existing background with it
uh I mean I haven't done this before I
don't even know if I'm really do what if
what we're doing would be considered
video processing we're kind of just like
taking frames and shoving them into
openg uh I mean I did work with security
cameras that had like video processing
to some extent in there um and I just
kind of like applied for a job and got
it so no no existing experience there
but I don't really know how much that's
like relevant here it's hard to know man
there's like cuz like what we're like a
sum of our experiences and so saying
like this didn't matter is like so hard
to know because you don't know like you
pick up up and so like just doing more
things gives you more context to do more
things and so maybe it's relevant maybe
it's not I don't know man uh but this is
new and it's fun so I don't know I don't
think like and nothing we're do we've
done so far like is requires any like
special knowledge I don't think it's
just kind of like poke and prod until
something works it's kind of the
approach that we take to everything
around
here um okay so how do we apply this
patch also I think I have like 40,000
Vim windows open just like kind of like
foregrounded on top of each other over
and over again so there we go I think
that's all of
them okay so I guess what we can do is
we can
uh we want to generate a patch between
this and that so it's let
patch how do you generate one it's been
a long time oh it's diff diff is the
tool to
generate and we diff from RE mini
audio hello oh uh diff from mini
Audio To Source mini
audio nope uh I guess let's try this but
I don't like this diff look maybe it's
because I'm
uh I'm too stuck on git where I get like
stuff like
this may but I guess this is fine so
we'll save this as a red a mini audio
lowercase U is that what I want it's
possibly what I want yeah that is what I
want hell yeah this is the stuff this is
the stuff um so s this as mini
audio uh
Zig where's our
Zig
18247 patch
18247 yeah okay and then in our build
script we apply the patch so we'll make
a function prepare mini
audio he's probably going to take in the
build
and we're going to run some system tool
[Music]
patch run
Sy here we
go here we
go
boom and we'll say tool run add
input where does the add input file Arc
oh it's just add file Arc add file Arc
so we are going to work
on uh
the mini
audio mini
[Music]
audio. this is not going to be JY this
is
patch and we're
saying
patch I think it's
uh Dash P1 to apply a patch and strip
the A and
B's what's in our patch
here uh I guess we don't if we just
delete the
prefixes right you can do this uh but we
could just do
patch so if I say
patch mini audio. Zig patch
honk
failed honk
failed uh
why uh okay let's try again so all this
goes away and we say
uh diffu and we'll now
do this
oopsies generate me a patch please and
then
[Music]
patch okay so that
worked oh did it patch it in
place
it did apply patch two stood
out d o dash okay so curl
and then
patch d o do Dash
cool and we'll say mini audio
2.h and then we'll
diff mini audio. and mini audio
2.h me
man okay so that did stuff
right this is a PL this version of mini
audio sure does have those
changes and so if we say hey could we
patch so we patch with
pum patch file original file patch
file and then that person was saying I
can specify an output file instead of
patching files in play
oh uh no I don't so
this patching file readed from mini
audio. and so here if we look for like
our stop
proc why is this not
fixed am I
high oh my patch is
broken that would do
it so now are they fixed
yeah okay cool cool that does it that
does it okay so we should be able to
remove these and then get add star here
just so that we don't get lost and now
we should be able to say
patch we add no a file argument of mini
audio mini audio. we add an arc that
says output to standard output
[Music]
and I think add output file ARG
redirects the standard output to this
file I
think uh cool and then how do we run the
tool as part of the
build install step depends on so
B get install
step.
depends on tool run step and so now if
we run Z oh we need to run this
function so we need to say uh pair mini
audio and if we do
this we should see our patched mini
audio show up in the zig cache
somewhere fine I'll type b. paath if
that's what what you really want from me
I'll do
it
um no do do I have it backwards it's
this
way no maybe I'll read the error I'll
actually read
it ah it's here it returns me a path of
where the thing God output I don't
really give a right
now uh expected type build step found
build step so I guess I need to put a
little Amber sand
here uh cannot patch
file this thing no such file or
directory
uh okay so what does this guy do it says
add file art oh output is tool run add
St oops I have to manually capture it
stood
out
and then they install it but I kind of
want it to end up just in the
cach can
I I guess I could just add the
argument oh I can just go um I'm an
idiot
I'm an idiot we just instead of
outputting to standarded Output we just
output to the file that we
wanted to Output to I'm going like why
would I do it that way okay so we add
ar- o and then we add the file that we
want to Output to then we run
it okay so in our cache now we should be
able to
find mini audio
Doh with our stop proc no
no me man
really oh I was so sure I was so sure
that was
it I was so sure not adding the patch
Arc oh yeah that's a really good point
yeah super super good
point super good point
so I think it's just like
this I
think okay let's try that so Zig cache
sum output mini audio and uh we can look
for ma stop Rock boom baby let's
go okay so now in our build
script um we should return the output
here
and we'll call this
a what is this a laaz path
yeah stood build lazy
path and then we can prepare mini audio
we'll say mini audio path is equal to
this and then
we'll pass these into here
then in here our include path needs to
include uh the mini audio path which is
a lazy
path hell yeah okay and then if we do
all of
that uh oh we don't need to
put a star on him we do all of
this then if we just type Zig
build I think we get something that
works
maybe I need it's Parent Directory
Parent Directory Parent Directory Parent
Directory does he have
a uh
uh can I get his parent name out of a
lazy path or do it like is that just not
possible uh relative durame yes thank
you
God
wonderful
maybe maybe b b
okay sick sick okay and then we just
have to make sure that we don't have
anything else kicking around in here so
let's nuke this
guy we don't need him
anymore uh this guy now should be
using these types of
brackets uh actually for the
implementation we want to use the
original huh
uh maybe it'll be fine I'm not really
sure if like the mismatched forward de
like the mismatched function pointers
are going to cause
problems they probably
will so let's do
this this will reuse the original Mini
audio directly and will use the patched
one uh res mini audio file not found
great
uh super cool so I
guess is there a way for us
to oh this is going to get so
complex so because of this like concern
about like type mismatch I want to
compile the original for this
imple and but this this like argument is
getting passed into the zig compilation
unit of like the whole exec so it's
almost like we want
to construct a static library that uses
the original with a different include
path just to dodge that that's
annoying uh but it is what it
is and I don't really know how to make
libraries I've never had to do
that never had to do that
um static
Library
so we just call ex. link library on the
library that comes out of AD static
Library
okay uh so we say B add static library
and we probably have to pass in uh uh
nothing maybe maybe this is fine what
goes in
here Mini
audio path uh root source
file is it going to be mad if I try to
make a static C library this
way would it not work to never use the
original feels like having the wrong
protypes anywhere in the proct is risky
um well that's what I'm trying to I I
well I guess we can find out we'll find
out uh it's pretty easy to test that huh
Let's test it and then come back to this
so if we go
to the zig cache output and look for
mini
audio.
H where is he
gone
here
zigs uh stop proc okay this is using
that
one and if we say uh GCC
uh Define mini audio
implementation on and then run mini
audio.
H how do I export a static Library
again uh what is the
GCC oh Dash oh test
Auto uh am I high GC oh J DC right
right oh it does look okay unless I st
mini audio implementation
wrong okay maybe I was just all up in
arms about
nothing yeah I was concerned but I guess
it's fine I guess it's fine
surprising okay so I deleting the cach
CU I was like with it
manually and I guess you're right we
don't have to worry about
this all
right
cool so that's uh an hour 25 and we've
done literally nothing super
sick uh
but at least now we can well probably
we're going to end up actually right
like two more lines of code and Hing the
same problem again that's actually
extremely
likely um but at least we know what the
problem
is at least we know what the problem
is okay so let's go back to trying to
get a little
farther down this
road so it was like
this uh this comes from CMA format 32
and we have our own data call back but
in this language we supposed to be doing
this uh we don't have any user data
right now and let's try uncom to this
function and hopefully it's all just
okay no field name c in enum
oh is it capital c here when I do my
calling convention might
be
uh expected
type this
found
type this so
found ah I need to change this to any
opic
now any opic any opic spe on thanks for
the raid appreciate it what's up you
guys uh for the new
people uh we were trying to work on
today getting audio call backs
working okay context we we instead of me
just telling you the like narrow version
let's look at the big version uh let's
see if this runs right now Zig build and
run okay so we have this like video
player right now he is taking frames
from ffmpeg and he's spitting them into
an openg texture that we're displaying
in a glfw window the reason we're doing
this is because we want to make a video
editor that makes it easy to like clip
and ship stuff so the idea is I'm going
to take my big ass VOD I'm going to run
it through some some sort of like
subtitle generator and I want to be able
to basically select chunks of text and
say I want this in the video or I don't
want this in the video haven't really
decided like which way we're going to do
it yet um so that we're we're working
towards that right now we're working on
the video player at all we need to have
a functional video player or else we're
not going to be able to interact with
this thing and so today we were going to
work on audio we pulled in this audio
dependency and we found a bug in Zig and
we adjust we added on to this existing
bug report with like a little bit of a
strip down test for how to reproduce it
um and now then we Now understand the
problem well enough that we can work
around it this problem results is like
related to uh function pointers to a
struct two structs point using the same
function pointer who is referencing one
of those two structs weird Edge case
doesn't matter um so we kind of got that
out of the way we hooked up to our build
system like we pulled in the dependency
patch of the dependency as part of our
build and now we're back to actually
trying to use the dependency again so we
are right now just trying to set up this
like audio call back we're expecting to
run into more problems but for now we're
just trying to get compiling and run
their
example okay so I think we're we build
uh let's just make sure that we're
building what I think we're building so
if I just introduce a compilation eror
he gets angry sick everyone's
happy and let's keep going so we have
some device who
was oh he's not the key is a VAR device
who was a c ma device
undefined and then we call c ma device
in it with null and our device and we
check if that's not equal to c ma
success and then instead of returning
negative one we will return some
error so we'll say uh failed to init
device and then we'll return error uh
device and I guess it doesn't really
matter this is kind of scratch code so
we're actually not going to end
up using this
directly we start the device and then we
unit the device
and that doesn't compile anymore uh
because this guy can return an error
probably uh I wish I could jump to
definition on there so we'll just
restart and hopefully that kicks the
language server into a state where it
works but no such
luck uh but we can just look at the
implementation here ma device start so
what's going on
here ma
result so presumably ma result is
something that can be true or false enum
ma oh enom type
def uh ma result here we
go uh yeah okay so we'll just say if
this
is not equal what what is going on oh
there's a comment in there stupid line
wrapping was tricking my dumb brain uh
if this is not equal to ma success we'll
say uh failed to andit start
playback
uhhuh
and it build we build and then I'm going
to mute the output from OBS before
running this because I don't know if
it's going to like up my ears
so we'll say Zig build
and
run our audio
test and we're mostly chilling looks
like uh he's mad about some sort of mem
copy
here no address available so I guess
there's a chance that we kind of
up with
our void star casting
it' be interesting to look at this is at
41
699 uh no he seems fine seems fine
probably uh so this is coming
from well did we something up else
up here
uh what is like ma device in
it so we have some device and we try to
initialize
him we try to device start and we are
crashing in device in
it LOL playing around with the zig bug
you found I got the compiler to se Vault
oh sick
congrats it's kind of
fun uh let's try GDB this thing
okay
oh uh is it because like I'm immediately
quitting am I supposed to be like doing
like while
true
run oh no okay here we go here's seg
fault okay so at
frame 4 we're calling ma device in it
with
[Music]
config there is a lot of invalid memory
in the
config uh I didn't zero it out would
they zero it out in their example oh
they called ma device in it we just
didn't do that so that's probably
the
issue uh CMA device type playback that
would do it
that would do
it and hopefully this still compiles but
it might not but it might we're chilling
Zig build
run uh oopsie Zig build and run the
audio
test okay so nothing's happening is
there a way for me to see if there's
like my system thinks that something's
happening um does this not show process
here I know I have this like widget in
the bottom right but it's like covered
by a bunch of
um I do like pavu
control I just want something that shows
me audio streams that are currently
active give him a second he's got to
think AVU control then do we have like
playback oh this is
oh no this is stream stimulator that's
this
guy so it doesn't really feel like it
did anything
huh if I run this guy and he just like
starts and stops that's fine um how do I
I think there's he might be using ala
also list
streams or like I don't even know what
sound server I'm using I'm using pipe
wire how do I check uh pipe
wire is pipe wire or pulse or alsoa
running how do I know what I'm like I
don't want to know which I don't want to
know like how to pick I want to know
just what the is running right now
in my system uh maybe we can do
like
PS everything grip
pulse grip okay this is
daming um
so let's see let's see let's see let's
see um how do we test this okay let's go
back to this never close the stream
thing and so now theoretically an audio
stream should be open what happens if I
try to add an audio
stream
in OBS audio capture device audio output
capture pipe
wire oh this is B device
me so I can't just capture like is there
like application audio capture in here
it doesn't look like there is oh
application audio capture here we
go uh application is anything in here
look like my thing audio test right
here oh but it might just be listing all
applications which also isn't very
useful we don't really
know you there's PW top
we're on pipe wire oh yeah nice okay
um we do see a mini audio thing in there
that goes
away okay that's a really this this guy
is in there for
sure sick okay okay so we have like an
audio
stream and I guess all we want to do is
we want to feed some data into it
so uh in playback mode copy data to P
output
so never process more than frame count
frames and what are my things I have a
device that I pull in output input and
frame
count so I don't know if they're
expecting exactly frame count frames
never process more than frame count
frames seems like they're kind of saying
I could process less but how would they
know how many I processed surely I have
to fill up the entire output buffer
right it's also kind of crazy that they
just give me output and input and
they're just like I don't know you
figure out which mode you're in that
seems crazy to me it seems like you
should have one call back for output and
one for input
but what do I
know what do I know
okay so is
there an example of
like
um we'll figure it
out so we know that our sample rate is
supposed to be 48,000
so if we wanted to play a first of all I
don't know what a reasonable sine wave
frequency is can we just like sine wave
sound one KZ sine
wave
that's uh pretty high can I go like less
than that like 440 HZ shut up oh my
God
skip yeah okay 440 HZ that sounds fine
so there like about like this over
100
uh so this is something like we're going
to have to put in samples of some format
we don't I don't even know the format ma
format
F32 let's do one channel for now just to
simplify things and so if we are at
48,000 samples per
second and we're trying to get
to we're trying to inject a 440 we'll
say 480 Hertz so this is a SPS s/s is
not helpful here so we'll call this
uh I don't know like item per second or
something it doesn't matter 480 is
seconds per
item
and S or
sorry this is items per second as
well so if we were doing it if we were
to take in like our
step um at each step we're going to put
in frame count things so maybe we'll
just stick in a globe right now uh like
samples
provided and let's say that they give us
like they say hey could you give us a
thousand
samples we should give
them we should give
them some time step of
samples so it's like a for Z to frame
count we need to do something right I
don't know what
yet we have to do
something we have some output which we
are we we're going to cast our
output is question mark St somebody said
question mark star any op is that is
that wrong here am I just supposed to
write any opaque and not question mark
star any
opaque no I think question mark star any
opaque is saying like I have a pointer
to something and I don't know what it is
I'm pretty sure
pretty
sure um and so we're going to do we're
going to do something with this
index so we have some
function that's like
sign sign
something okay let's just say Let's
ignore the sampling stuff say we want to
generate a 440 HZ
wave sign normally goes
from 0 to 2
pi so if we want this is one period so
if we wanted one sample per
second one if we wanted one full
movement of the sign wave per
second we would
do
s time step time time step time 2 pi
because then when the input is if time
step equals 1 then we get sin 2 pi which
gives us
zero yeah yeah yeah yeah yeah yeah
okay and then if our sample rate
is
48,000 then we just divide this by the
sample rate and then multiply it by
the desired high
frequency oh I also forgot to unmute the
desktop audio when I was playing sign
waves for you guys earlier but this is
what I was trying to get
out um okay okay okay okay I think this
is right not really sure
but we'll find
out uh so we'll say time art samples
provided plus equal I and then we will
just say that uh our output as a float
as
floats is going to be some sort of
like slice of
F32
maybe and that's going to come
from I guess we'll start by saying that
it is a
uh array pointer of
F32 and this is going to be the pointer
cast of
output then we can say output F at I I
don't know if this is valid Zig syntax
but we'll find
out is
uh at
sign samples provided times stood math.
Pi * 2 / the sample rate
time desired
frequency
and maybe it
makes oh it's fine it's
fine and here we will do as F32 float
from in of this thing
uh yeah
sure I don't really know if that's right
but pointer cast increases pointer
alignment so we have to say align cast
here and uh I'm going to mute the audio
again before running this that you guys
don't
die uh I'm hearing nothing nothing which
is not
good uh not good it's also kind of
surprising
um I wonder if this is this code
running I guess we can find out by
running uh
GDB uh yeah it is
running okay and we have 480
frames here okay
and it's there there's lots of things
that could be going wrong here to be
honest lots of things cuz we're kind of
just guessing what's supposed to happen
um it would be interesting to maybe
stood debug
print the array that
we are returning which is output from uh
as
floats from zero to frame count let's
just see if that looks like anything
saying Zig build and run the audio test
C
dump cord
dumped
okay
okay let's maybe before we do this print
the uh frame count is going to be a
frame uh a number let's just see if this
is something interesting
uh Hello computer jeez he's mad at
me I'm indexing a finite buffer with
infinitely growing samples provided I
don't think so I don't think so samples
provided is only an input to our sign
function it's only input to our sign
function so I think it should be
fine uh I guess we don't really
know Ma format at F32 32-bit floating
point in the range of netive 1 to 1 so
sign is giving us should be giving us
netive 1 to
1 uh which I do believe is
happening okay so there he's not
crashing and he
is giving us
something it's could very very likely
that the math is wrong here this stuff
looks like it's moving pretty
quick uh
so oh but I guess it should move pretty
quick let me think let me
think um so we could
do let's let's double check this output
math
again uh
so
we have some number of samples
provided say that this number is
48,000 what do we get if we plug that in
here
48,000 time stood math pi
over
2 St time or time stent math Pi * 2 by
48,000 * 440 that gives us 48,000
cancels out with
48,000 and then we get 2 pi * 440 which
is the number of iterations that we
wanted in 1 second so that all feels
right to me
that all feels
right is there like something about like
volume of the stream that's coming out
[Music]
maybe let's get rid of these prints
because I'm sure that's clogging up our
call
back in capture and in full duplex mode
these will be valid and you can move
data from P input to Output never
process more than frame count frames
samples R plus equal I should be plus
equal 1 that's a really good point
that's a very very good
point I guess the fact that occasionally
we seg voler is a really good indication
that we're doing something
stupid right really good
chance uh so maybe let's look at the
mini audio. again and look at a well
maybe they have examples on their
website
actually maybe start starting there
makes uh simple capture simple playback
oh wait this is exactly what we want
simple play back
line so they
make
ma re waveform read PCM
frames ma a waveform config oh they're
using uh the
the highl version I
think oh maybe
not so here they're saying start this
device and we got this device by doing
this then they Loop they wait forever
while the audio thread calls into their
data call back okay and then the only
thing that we're they're doing that we
aren't is they're doing
like ma waveform read PCM frames but I
feel like we should be able to shove PCA
frames in directly what is that Ma
waveform read PCM
frames yeah this is generation of stuff
but I don't want to use their generation
I want to I want to feed in my stuff
directly but I guess we could start by
trying
that so that we have like a
sane
sample Z's a of B syntax is not
inclusive is it no I don't think so oh
so you think this should be frame count
plus one I think that if even if we were
off by one here the difference like we
should still be hearing something that's
like not
nothing uh if that makes
sense also this Segal is like a really
good indication that we're doing
something
wrong um
so what I wanted to know was what the uh
frame count
is in the above example data call backs
where audio data is written and red from
the device the idea is that in playback
mode you cause sound to be emitted from
the speakers by runting Audio data to
the output buffer in capture mode you
read the blah blah blah I don't care
about capture mode right now the frame
count parameter tells you how many
frames can be written to the output
buffer and read from the input buffer
a frame is one sample for each
Channel One sample for each
channel for example in a stereo stream
one frame is two samples
okay the channel count is defined by the
device config okay multi Channel audio
so I guess there's there's a few things
that could be going wrong here
um maybe the the config we set isn't
necessarily the config we got
um
and what was the other thing that I
thought it might be I already
forgot uh oh yeah I thought that maybe
it's like
stereo it's coming in Stereo and we
don't expect it to be you're saying like
maybe minus one I don't see why minus
one would ever be what we need
here but me
check um
so before we were seg
fating when we tried to print out the
output so maybe it's just like we were
taking we are we were backing up the
audio for the audio call back thread too
much that was causing problems cuz
writing this isn't causing any
problems uh let's what happens if I go
back
to two channels
here o okay that's
interesting so that's something sorry
your guys ears are probably like
freaking out
let's okay so it's possible that one
channel did not give us something sane
so if we do two channels this should be
easy where we just go Um for 0 to
two we do this I'm pretty sure this
should be like this
still not good still not
good uh right so uh I plus
J still not pleasant on my ears at all
at all that's not like too loud right
now after I turned it down right that's
like fine for you guys I'm assuming it
is but I don't want to like ruin you
all's
ears I guess nobody people aren't
screaming ow in chat which really
indicates that it's okay
okay
so for each frame in frame
count we are going to I equal 0 and I
equal 1 assigning oh this should be uh I
* 2 there it
is
okay all
right not bad at
all okay so I think we get it now so I I
guess the question is why when I set
this to
one was this not
working so here let's uh split out a
cons num channels is equal to two what
like type
of what type of this is this I guess
we'll find out real quick if I try to
write usze here I guess I'll just leave
it as a comp time and then we can set
this everywhere num channels is here and
here and
here
so I don't understand why if I set this
to
one we just get nothing
back it's possible that we just uh if
that the initialization gave us
something different can we look at uh
maybe we can look at our device in
here so let's look at um con
device we'll call this device V for void
we'll call this device is a
uh CMA device pointer and so we'll say
this is please uh pointer cast align
cast this device V that came
in and
then let's look at what's in there so uh
ma
device struct and a device here we
go
so we can look at his
uh sample rate does he not have numb
channels in
here oh here struct
playback so we can look at the uh device
playback
uh channels
I guess maybe it makes more sense to
just break point here and look at
it uh but maybe we need to print
something from it so that the debugger
has something to look
at so here we'll print the number of
channels but then we'll set the
debugger okay so here he does see one
channel
and can we uh GDB this thing I guess we
need to yank here GDB break on
this
run and uh what's in here P
device
Jesus
playback device
is it a p device Arrow playback does
that work here it
does it
does so we do have a
48k sample
rate and single
channel so I don't understand why if we
weren't in Stereo mode doesn't work
maybe it's picking a different output
device if like I don't I don't know what
this like device finder is doing to be
honest
I have a FFM pic mini audio example I
madean a while ago if you want uh that's
cheating no
cheating however I appreciate it but I
think it's more fun to struggle through
it this
way so if I have like multiple audio
output
devices right like I have an HTMI HTMI
app
this like sink and like a line out how
do I know which one it
chooses that's the only reason I can
think of that this wouldn't be working
when I when I go to one channel because
it seems It's like we have as far as I
can tell a valid
device I wonder if I can like print the
name of the
device uh
ma a
device ma device struct ma device what's
in this
guy device
type are you sure the naop is the same
as a sign upos for both single and
double
Channel there was a type enum in the
playback I think to see if that
changes
um are we sure is the same
type so you think
like if the in single Channel he might
have not chosen
floats that seems possible uh we have in
here the sample format somewhere
internal format five so can we look at
what
uh ma format F32 where is this
defined
five so I don't really
see I don't see that it would be a
problem here CU if if our structure here
is telling us that it has format five I
don't see why it would be
lying null playback device that'll
do
it that will do it so I bet you
if we turn this back to two
uh and then we run this again and run
the same run it with
GDB I need to yank the line again
uh it was
here oopsies what am I doing uh GDB bre
here
run and here we should be able to say
print device playback
and here it's default playback device
instead of null
so whatever the audio library is doing
when we ask it to initialize device he's
like looking for the best device and
then the best device for that
configuration was
null which is interesting it kind of
seems like we should
um like like what is like the sane
Behavior here the same behavior is that
the user says which audio device they
want or maybe not the user right I guess
there's a we should ask the system what
the default is and then we should force
it to use that and if it can't
use that then we should fail and ask the
user if they want to use a different one
that seems like the most sane thing um
which makes me think that in the
config there should be some way to
specify which device we want
why is it to succeed if the device is
null yeah I agree I I
agree uh okay so device
type like what are any of these things
like the device that we want ma device
ID this seems like what we
want so ma device ID
has some
Union it's a union of
which backend to use where if it uses
ala it will give a name or if it uses
pulse it'll use a name so that means
that here how does it know which version
of the Union to be
using right so if this thing's a union
how do they extract the right piece out
of
it you
know or it's a union
ma Union
Ma I lost
it oh this is oh sorry ma device info
is not the union ma device ID is the
union and
so somehow they must go
from do like do do they not have like
some sort of you like tag I guess we can
look for
um
ID
doala and figure out how they get
there I see so they know they know which
back end is being used and so that they
know which one of these guys to look at
depending on the back end that's in
use so here probably somewhere we can
iterate these like device
infos simple
enumeration yeah yeah yeah so they we we
can say hey could you list the devices
and then we will pick
one and
then
uh we will force you to use that in when
we configure the device that makes sense
so
far ma context get devices as well see
also ma context get
devices it is not safe to assume that
the first thing is the default device
sure but how do I find the default
device
if you have multiple physical devices
you want to use a specific one you have
to specify the device ID configuration
yeah okay to retrieve the device ID oh
cool so just uh like maybe like six
lines down from where we were looking
before uh it's going over exactly what
I'm confused
about uh so they get the devices and
then they Loop over the device they put
them out and chosen player but how do
they how do they know which one's
default
um if you have multiple physical devices
you want to use a SP one like like the
system has one selected right like if I
go into my audio
devices how come it was there a second
ago not now okay like this is selected
so what does this do this must go into
pipe wire and flag it in some way and
also I think that it's also um
is it alsoa
mixer Oh wait we're probably using it's
using the pipe wire the pulse audio back
end yeah it has pulse audio back
end so he must ask pulse
audio he must ask paulse audio somewhere
what the default is
so how do we go from uh device
enumeration
to enumerate devices pulse
okay get default device index pulse okay
okay okay okay okay this is
good and then what does he do with this
he puts this into default device index
playback which
is must be used somewhere device default
device info is
default so I just didn't read this whole
struct so we iterate these guys we check
which one is the default and we just use
that one and we force it
and if that doesn't
work then hopefully this will fail okay
okay okay so we can go back to their
example here where they iterate the
devices
so we don't have a context yet I don't
think no we don't so we must just have
to make one I guess before we do
anything
else okay uh undefined not
null then we try to initialize it
so we will copy paste this and we will
call this context with
null0 null and context I don't know what
those do but that's fine and we will say
F fail to init
context we will say hey we have some
infos which is
a I guess it's going to be a list of
infos that we could
get no he's going to allocate this for
us he's going to allocate this for
us it looks like which kind of indicates
that we're going to have to free those
things but he doesn't say that so
probably they're it's going to be part
of the context and we're going to have
to when we free the context all of that
stuff gets freed
probably that's just my guess based off
the docs but I mean I've made a lot of
bad guesses today so you know who
knows uh so here we will call this a c
pointer to a CMA device info is youal
undefined and we'll say uh num playback
infos is going to be a CMA Unit
32 also
undefined and then we will call this
function
with uh the context the playback infos
playback count these guys are going to
be null and
null the null device probably exists to
have a fallback on a system with audio
Hardware it just seems crazy to me that
your default would be to allow using the
null device though like that's just like
really unexpected Behavior to me like I
would think that you have to
specifically opt in but I like sure I
get it like it makes sense to have it as
an option so that like you can like
fulfill your interfaces and stuff and
just I'm surprised that it works by
default I guess probably there's a way
to turn it off
H I think I saw that they have like back
ends that they that you can uh
yeah this probably would have helped us
but it's too late now well it's not too
late but I don't care I guess I want to
keep doing this what we're doing because
I think this is more correct
anyways
um so here we'll say failed to
get
devices and this is playback
infos and this is uh num playback infs
like
this and this is c ma sucess and now we
should be able to say uh for playback
infos from zero to non-play back infos
we have an
info and then we're going to say
uh VAR
[Music]
selected device is equal
to0 uh which is probably going to be a c
ma
un32 and then we're just going to say if
info. is
default selected device is equal
to I guess we need to
do take in an index here
which kind of makes this a u
size and then down here wherever we set
the device that we
want I think we had to set that in the
config right yeah
[Music]
config
playback P device ID is equal to our
playback infos at selected
device
boom so hopefully this all just still
works uh let's make sure our num
channels is to it is so Zig build and
run we're not good because it has no
member on in it uh right of
course this should be an underscore
and uh this should
be uh no this should be context here I'm
just
high high as I
guess uh found you for is
default so we'll just do a little not
equal zero there P device ID no field
name key device
ID is it oh capital
D
fortifier of
course uh and then this device
ID is I need to write do ID
here so they want
that okay so that's still working with
two channels and now if I do one channel
what I hope is that this will fail
no it just so
off so here do I have to assert now that
the device that we
got is the one that we asked
for
um no failed name p device ID on people
on playback of the
device ma
device
struct M device all
right so here we have our playback
section here and does he not have the
device ID in here key ID of course of
course why would it be what it was in
the other
one okay so this now a asserts because
those things are not the same and if we
set this to two it probably will assert
that they're
true no oh interesting what is the
difference Ma device
ID key device
ID set to n using default ID otherwise
set to the address of
ID uh maybe this is what I want ID and
this uh oops this needs to be like this
we dreference on the other side in this
language comp comparator not allowed for
ma device ID
great because this is some complex
type of course of course
try to find mad bug
output uh yeah I guess that's not a bad
idea
um so I guess this be in the imple here
Define ma debug output probably
uh oops I can't do that comparison
still assert where is my assertion get
this out of
here okay so that's two
channel one
channel okay so here he said okay I
still ended up on the null playback
device that's not too helpful not super
helpful
oh struct ma
device it's crazy to me that I guess we
should just look at what the device idea
is
I guess we should just uh
actually debug hit the
debugger and probably in a new window
actually uh Nick shell
oops let's get GDB in
here give him a sec he's got to think
he's got some stuff to get from the
internet but it's not actually from the
Internet it's actually from the hard
drive I don't know what he's doing in
all this time but it's okay
uh okay I don't care about these e
events okay and if I say hey can we look
at the device
please uh this is a lot to take in E
device playback
ID uh oh we know this is sorry let's go
back to the one where it's working I
want to look at the working example not
the broken example
Zig
build okay GDB break
here uh run I have to
oopsies okay EO e e e EO P device P
device playback.
ID uh
might be null
here to be
honest uh P ID sorry that's what I
want uh which doesn't really seem to
have anything that interesting in
it even though it was working right this
is the functional
case what did our config say p
config playback
P device
ID okay so this is also kind of just
giving us zeros
here which doesn't quite feel right to
me what else do we have in here we could
print the name of this guy
uh so we could look
at we could look at the playback
infos at selected
device and hopefully we see is default
ones that's good and it's got the name
default playback device great
and if we look at our the device that we
have can we get his name is that
possible okay so it does say it here
wherever the this
is uh which is in custom
somewhere jeez why is this formatted
this way what an awful way to
print
stuff so this is next to ID which is in
playback yeah yeah so
device
playback name okay yeah here we go you
have name and ID so in this
scenario set print pretty is it not I
thought I had that on by default oh
that's so much better thank you I
thought that I had that somewhere in a
config and then I I thought that yeah
I've looked at this before I just lost
it thank you thank you thank you thank
you okay so in this scenario we got the
device that is default oh can we just
check if
uh device playback is this does it have
is default no it
doesn't he device does he have like uh
is fault anywhere
no so I'm a little confused about how
the device
ID uh device playback
ID this is just like hard
zero right this is showing us a union
where everything in it is zero and if we
look at the config
playback key device ID and we do
reference this
that's also like hard zero so that's
fine by me right I don't have a problem
with that but if we go to num channels
one and we do the same
thing give a sec give a sec give a sec
set print pretty and we say print
config uh
playback key device ID and we do
reference this it's like again hard zero
right like super hard zero and
if we look at P device playback ID it's
like hard zero so it's like what the
is going on here how come we can
have the same thing in these two
situations and have them just be like
completely
different it's possible that the pulse
audio default just is coded to
zero
and then the it switches Behavior based
off of what's
available that seems
possible um so it's right right right
right because we saw the name is default
audio device in the config right if we
look at
um what was it called it was P playback
info
uh selected
device his name is just straight up
default playback device so it might be
that it's doing something stupid like
pulse audio might be doing something
like a little bit silly under the hood
and uh wait hold on P
device no he does still say null
playback device which is a different
thing I wonder if like our
config maybe has to say like don't
fall
back here they're not really saying to
do
that um ma device config is that what
the thing we were setting yeah it
is is there like a fallback flag in
here I'm kind of hung up on the fact
that the device IDE is the same
here but like this device ID is like
hard useless it feels like
um where was
it where was
it capture playback right like the fact
that this ID is just pulse
000000 like what the
I wonder what happens if you define
ma no n that's a good question do I have
to Define this on both cases or just on
the impul
side I think probably we'll try here
Define ma no no I am curious what
happens
there he's working hard oh failed to
init
device okay and then here the debug is
saying null back end is
disabled
oh he's
probably failing to initialize pulse
audio and then falling back to
null so so the the behavior of us saying
hey we want this device in this
configuration and pulse audio
goes
no and then we give
up so where does this initialization
happen I'm not really I'm a little flaky
right now
[Music]
um so if I go here and I
say stood debug print initial in
initializing device it we'll have
the caps in there because it
will reflect how angry we are right
now where does that happen initial
device
here so back
here it actually failed to initialize
pulse
audio which is a little surprising to
me how did we enumerate the devices if
he couldn't load pulse
audio oh cuz we're we're using Jack
probably
okay cuz I know that um pipewire exposes
both a pulse audio and Jack back end so
that must be what's
happening but then it doesn't really say
let's see what happens what it what it
does when we say num channels
2 let's see what it does
different yeah okay so he initialized
the Jack playback device and it just
must be that when we set it to one it
fails and going back to the null device
as a fallback it's just what a s what a
weird what a I don't like that
decision so I think compiling with ma no
no makes sense for us right because we
want to fail in that
case so I think that makes sense I think
that makes
sense okay we're at 2 and a half hours
so I think it is time to call it um than
thanks for watching guys we could didn't
really do much of what we tried to
accomplish today we did a lot of uh like
probably the part where we were like
actually attempting to play a sine wave
was like arguably not that much right
maybe like 15 minutes of like actually
trying to set this up but we had a lot
of fun uh debugging the zigg C in aop
where we were ending up with some
invalid Zig uh triggered by Min audio we
found a good workaround for that we
committed a patch to our repo that will
Auto patch mini audio on the Fly uh
during build which is pretty cool pretty
fun um and then we updated the bug
report there somewhere do you have pipe
bar also install probably not probably
not uh the fact that it's ending up at
Jack kind of indicates that it's like
skip like it failed to load pulse it
failed to load ala so it just must be
something is there's it's Nick some Nix
uh yeah and then we got our S
Wave going and then started looking into
like why mini audio was not behaving the
way you expected um and just related to
him trying to Output to like if we give
it a config that we don't like that it
doesn't like then he just like gives the
up and so I guess there's something
about we should be like walking the
configs in a certain way and I guess it
also means that we should be handling
when we play our audio back something
related to like if if if the audio is at
like some sample rate that we don't
support we need to like try to find the
best sample rate that's like the closest
to what we support to like what the file
is and then do some sort of remapping
probably right like if we have mono
input and stereo output only then we're
going to have to like duplicate the
signal stuff like that so got a bit of a
feel there um we'll probably just run
back at it tomorrow slam our heads into
the wall some more and uh yeah thanks
for watching guys uh if you like what
you saw there's a YouTube link and a
GitHub Link in the twitch description
YouTube has uh playlists for all the
projects we've been working
on so I always forget what was it what's
in there so if we just look at them
right we have like video editor QR code
decoder zip get gooey TV tracker neural
network uh operating system stuff like
that file systems terminal emulator so
we Trend towards uh lowlevel stuff but
generally just kind of like whatever the
I want to do and that just happens
to be something that some people want to
watch as well
so that's all there if you want to check
it out um if you're watching on YouTube
there should be a twitch Link in the
YouTube description and I think that's
all we got to plug so uh by YouTube
viewers uh twitch viewers let's figure
out who we want to raid