Welcome to module 3 of digital signal
processing.
We have just seen the basics of signal
processing.
Mainly signals, operators, and examples.
We now setup the mathematical frame work
so
that we can study signal processing in
detail.
In model 2.1, we start very slowly by
introducing.
Intuitive geometrical notions in the real
plane, or so called R2.
In models 3.2,
we present the main characters, which are
vectors,
norms on vectors, inner products, and
bases for
spaces in particle for Hilbert spaces
which is
a natural generalization of Euclidean
geometry to higher dimensions.
Finally models 3.3 we consider
approximations.
Approximations are very important in
signal processing for example
for filter design, for lead square
approximations, and so on.
Let us get started with a review of the
geometry in the play.
This is very simple geometry as we know
from.
Great school exactly called Euclidean
geometry and
the key is that the ingredients namely,
vectors in the plane are enough
instructions
that is very useful to represent signals.
And so we have an intuition from
two-dimensional geometries that
we can generalize and manipulate signals
later on very easily.
The key insight
is that signals indeed are vectors.
Vector geometry is a natural setting.
Notions like orthogonality are very
general.
Now, this intuition is, of course, very
old.
It goes actually back to Greek geometry,
and to Euclid.
And Euclid in his famed book called the
elements actually
wrote down the basics of geometry more
than 2000 years ago.
We can see this in a beautiful facsimile
copy
of Euclid's elements, and for example the
Pythagorean theorem is proven in a
geometric way.
And this very theorem is actually called
Parseval
theorem, it's very important in signal
processing in general.
The module is split into three parts.
The first one is a
very lightweight introduction to signal
processing seen as geometry.
And that's really the Euclidian view of
the world, and how we try to go from that.
Very classic view to Hilbert spaces.
Module 3.2 is more formal, defining
vectors, vector spaces,
inner products, and finally Hilbert
spaces.
Module 3.3 discusses the construction of
bases
for Hilbert spaces in particular,
autonormal bases.
Let me make just an introductory comment.
So usually when we go to class and we talk
about Hilbert
spaces, people are sort of worried because
it sounds abstract and mathematical.
The goal here is
to show that Hilbert spaces, which is the
frame work
to think about signal processing, is
simply a generalization of the.
Usual Euclidean geometry that we know.
The spaces we live in.
The three dimensional volumes of the
physical space we inhabit.
Or the four dimensional space when you add
time.
And so it is nothing to be scared about.
Nothing forbidding.
It's actually something very natural.
And we're going to do this by entering
this field very gradually.
And we first start by putting an origin
here.
That's a black dot, it's the origin.
Then, we can draw a vector x that
has two coordinates, x0 and x1.
Remember, our vectors are column vectors.
So when we write this, we have to put a
transpose.
We can write the length of this vector
which is the distance from the origin to
the
tip of the vector and that we write
with this double bars to indicate the
square norm.
Then we can add a second vector in the
plane, coordinates y0, y1.
It has
a length given by the norm of y.
Once very have two
vectors, very naturally there is angle
between the two vectors given by alpha.
Okay, so that's Euclidean geometry in R2.
There is a particular configuration when
the vectors have a right angle.
So when alpha is equal to pi over 2,
sometimes this
is written here with a dot or with
annotation like this.
This shows that's a two vectors or
orthogonal to each other.
Now, the point we want to emphasize here
is that vectors can be very general
objects.
So, one set of vectors that is
interesting, it's functions
over continuous time like here, this sine
wave or another sine wave.
For example over an integral.
Here's integral minus 1 to 1.
And we can define a space
of all the functions over this integral
minus 1 to 1
that have a square integral.
So if you square the function and
integrate it
over this integral it's going to be a
finite value.
And here we have two examples.
So, x1, which is sine of f1 of t.
X2, which is sine of 2 of t.
And we can decide to calculate what is
defined as the inner product.
Which is simply the product of the two
functions,
integrated over the interval from minus 1
to 1.
Okay, so this is very important options,
that's the inner product between x1 and
x2.
What is to be noted here is that
previously we had vectors in R2 with an
angle
alpha, with an orthogonal angle of 90
degrees, here, we have also two vectors.
And the question is, can we have the same
notions, okay?
And sure enough, if we multiply the two
functions together, we get this red
function here.
And if the two frequencies, f1 and f2.
Are
integer multiples of a fundamental,
harmonically related frequency,
it would turn out that these two vectors,
these abstract
vectors x1 and x2, will also be orthogonal
to each other.
We're going to calculate this, okay?
So we take the product.
And then we look at what is positive and
what is negative because we are going to
integrate.
And we sort of see that for every positive
part, this one, we have a negative part.
This positive
part we have a negative part.
This one we have that one.
And the same here and there, here and
there, here and there.
And so when you take the integral between
minus 1
and 1, the integral is going to be equal
to 0.
So conceptually we have exactly the same
phenomenon as the two orthogonal vectors
in R2.
Okay, so that's the fundamental message.
Vectors are very general objects.
But they still have relationships like
length, or norm.
And angles between them.
And that's the magic, both of Euclidean
geometry and of Hilbert spaces.
Okay, so once we have a space, we can
define bases for this space.
The simplest one is our tool, we're still
in our tool here.
Is an orthogonal basis with a
first vector here e0, a second vector e1
and a right triangle between the two.
Any vector x can be written as a linear
combination here of e0 plus a linear
combination of e1.
Summing together, you get the vector x.
Okay?
A more general basis, still in R2 is a
biorthogonal basis.
So we have a vector v0, a vector v1.
These two basis vectors span R2 but they
are
not orthogonal to each other but we can
obviously
write any vector x as a linear combination
of v0 here plus v1 and that gives us x.
Okay, so what is a basis?
It's a set of vectors that spans the space
of interest, and that
is linear-independent, so it's a minimal
set of vectors that span, for example
R2, in this case it's two vectors, if we
were in R3 it would be three vectors,
etcetera.
Okay.
We can have sometimes too many vectors for
the space.
So we are still in R2.
But we have now three vectors.
So these guys are not linear independent.
We're going to show this just in a second.
But it is clear that we can write any
point in R2 as a linear combination of
these vectors
because of course, two of them are already
enough.
So thirdly a third one.
Can easily be ignored, or can also be
helpful.
Okay, now let's see that these guys are
not linear independent, by actually
verifying that a sum of the three vectors
is actually equal to zero.
So let's sum x1 plus x2 and finally, minus
x0 and you see that.
X1 plus x2 is equal to minus x0, and so we
can verify this equation here.
So these guys are depending on each other.
Which is of course obvious because, for
example, we can very well write
in this case, it's x0 as a linear
combination of x1, or x2, etcetera.
Okay.
So that's the case where the vectors are
not linearly independent.
If we have instead of too many vectors, we
have not enough vectors for the space of
interest.
So for example here, we have R3, three
dimensional.
Physical space, for example, but we have
only two vectors.
Then, we cannot hope to describe any
vector in R3.
We can only describe its projection onto
the
subspace spanned by e1 and e0 in this
case,
so the vector x cannot be exactly
represented.
We can actually only project it here to an
x at.
This far character here, which is the best
least squares
approximation of x on the subspace spanned
by e0 and e1.
Fill in the
[UNKNOWN]
of subspace, which is, let's say it's a
horizontal plane in this case.
Now interesting questions appear when we
go to spaces of infinite dimensions.
So we look at the interval here of minus 1
to 1, which is a continuous interval.
And so the intuition is that, there is no
finite dimensional basis
that will be able to represent an
arbitrary function on this inverval.
So let's look at the particular basis.
It is built up from so called sine
x over x or, more precisely, the n's basis
vector here is given
by sine pi n t over n, the final gentle
minus 1 to 1.
And we look at the summation of adding
more and more of
these basis vectors.
So the summation goes from k is equal
to 0 to capital N so it has n plus 1
terms.
And we look at the vectors with index here
2k plus 1.
Okay.
So when capital N is equal to 0 we have
one term, it's a sine wave.
Then when we go to N is equal to 1, we
have two terms.
We have the addition of two vectors, N is
equal to 2.
We have three vectors, we start to see
something interesting happening.
Okay we still a very smooth function.
N is equal to 10.
We start to really see what will be the
result or approximately the result which
is a so called
square wave, so something that will be
close to minus 1,
close to plus 1 and has a transition here
around the origin.
Okay?
That's for N is equal to 10.
N is equal to 50.
Well, it looks more and more like the
square wave, but it
has these little wiggles, okay, in
particular at the points of discontinuity,
okay.
And, N is
equal 150, it looks very close to the
square wave, except at the boundaries.
So we see here something very particular
happening.
It's called non uniform convergence.
It's a term from Fourier theories, theory,
that we'll not investigate more.
But it's something that is known, also, as
a Gibb's phenomenon.
And is an annoyance here in this
approximation of this.
Square wave or box functions by these sine
waves.