>> Welcome to module one of Digital
Signal Processing.
In this module we're going to see what
signals actually are.
We're going to through history, see the
earliest of examples
of discreet time signals, actually it goes
back to Egyptian times.
Then through this history see how digital
signals for
example, with the telegraph signals,
became important in communications.
And today, how signals are pervasive in
many applications
in every day life objects.
For this we're going to see what the
signal is,
what a continuous time analog signal is,
what the discreet
time continuous amplitude signal is and
how these signals
relate to each other and are used in
communication devices.
We are not going to have any math in this
first module,
it's more illustrative and the mathematics
will come later in the class.
>> Hi, welcome to our Digital Signal
Processing class.
In this introduction we would like to give
you
an overview of what digital signal
processing is all about.
And perhaps the best way to do that is to
consider in turn what we mean
when we use the word signal, when we use
the word processing or the word digital.
And you will see that digital signal
processing is
really an intermediate point in a
reflection about physics,
about math and about the reality around us
that started
a very long time ago and continues to this
day.
So let's consider the concept of signal to
begin with.
In general a signal is a description of
the evolution of a physical phenomenon.
This is best understood by example.
Take the weather for instance, the weather
is a
physical phenomenon that we usually
measure in terms of temperature.
So temperature becomes
a signal that evolves overtime and that
represents a measurement of the underlying
physical phenomenon.
We could've chosen another variable.
For instance, we could've chosen rainfall.
that would constitute another signal
related
to the same underlying physical
phenomenon.
Another example, easy to understand is
sound.
Sound can have very many origins take for
instance some musical instrument or a
person singing.
Now when you measure sound
with a microphone for instance, what
you're measuring is
the pressure, the air pressure at the
point of measurement.
The microphone translates the air pressure
into
an electrical signal that represents the
sound.
Now if you want to record the sound on a
magnetic tape for instance, you will have
to convert this electrical signal to a
magnetic
deviation that can be impressed over the
magnetic tape.
And again, these are different
representations of the same underlying
physical phenomenon.
Taking a photograph is a very similar
operation.
In this case, we're mapping the light
intensity of a scene onto gray levels, in
the case of a black and white
photograph, that can be recorded by
photographic paper.
The only difference is that in this case
we're
mapping the signal over space rather than
over time.
To make things more tangible let's go back
to an
experiment that most likely you carried
out in elementary school when you
first learned about experimental
procedures and
analysis of the world around you.
You were probably asked to map the daily
temperature for say,
a period of a month, and to chart it over
graph paper.
And so you dutifully looked at the
thermometer every morning and then at
the end of the month you probably ended up
with a graph like this.
So here we have two concepts
that are fundamental to digital signal
processing.
The first concept is that the temperature
measurements are taken at
discrete moments in time and they
constitute a finite countable set.
And the second similar observation is that
the range of temperature is actually
subdivided into
a finite number of possible values which
are
determined by the resolution of the
graduating scale
on the thermometer.
So, we look at the height of the mercury
column
and we chose the tick that is closest to
that level.
But, nonetheless, the number of ticks that
we can chose from is finite.
So the two fundamental concepts here are
the
discretization of time due to the fact
that we
take observations regularly but not
continuously and the discretization
of amplitudes due to the fact that our
measuring
device has a finite resolution.
Now the discretization of amplitude is
usually treated as a precision problem.
We can use more sophisticated instruments
and achieve a better precision.
But the discretization of time is a
veritable
paradigm shift in the way we think about
reality.
So much so that the problem appeared for
the first time over 2500
years ago when the great Greek
philosophers started to think about
why is it that we perceive reality the way
we do.
And the first character in the story here
is Pythagoras, who in 500 BC, maintained
that
most of reality, if not all of reality,
could be described in terms of numbers and
measurements.
Think for instance of the Pythagorean
theorem,
if one draws a right triangle, one can
verify experimentally with the ruler that
the square
built on the hypotenuse is equal to the
sum of the squares built on the sides.
But what Pythagoras said, is that this is
a universal
property that applies to the abstract
class of all right triangles.
And this was really a major change in
the way people started to think about
abstract concepts.
Pretty much the same time Parmenides,
another character in our story, brought
this
line of reasoning more into the waters of
metaphysics by planting the seed of
a fundamental dichotomy, a fundamental
difference between
the reality that we can experience with
our senses and an ideal reality that we
will never be able to know.
This of course was later developed by
Plato into
a full fledged philosophical theory of the
ideals.
So that even today when we talk about the
Platonic ideal, we refer
to some form of perfect reality that lies
beyond the veil of appearances.
But more interestingly for us is the fact
that right when this idea started to
appear in
the global consciousness of the time,
there were philosophers
that were ready to point out the potential
pitfalls
of this new abstract models of reality.
And the leader of the pack, so to speak,
was Zeno of Elea whose paradoxes have
survived to this day.
One of Zeno's most famous paradoxes is the
paradox of the arrow, which states that if
you shoot an arrow from point A to
point B, the arrow will never reach its
destination.
And the reasoning goes like so,
well if we modeled reality with the
concepts of geometry then
we know that any segment can be divided
into smaller segments.
So what Zeno said was that the arrow,
after leaving point a, and before
reaching point b, will have to travel
through the mid-point between a and b.
Let's call this point c.
But now after it has reached the midpoint
between A and B
it will also have to pass through the
midpoint between C and B.
Let's call this point D.
And so on so forth, for every interval you
can always find an extra midpoint
and the arrow will have to cross
all of this midpoints before reaching it's
destination.
But because of the geometric modelling
of reality there is an infinite number of
midpoints and so Zeno said, well in order
to cross an infinite number of points, you
will need an infinite amount of time.
Of course today we rebut such an
argumentation
by saying simply that we can express the
length
of the segment as the sum of all the
sub segments and that this sum converges
to one.
But this is a false answer to the problem
because the problem was never with
computing the sum.
The problem was with a model of reality in
which the infinite and the finite are at
odds.
And it took over 2,000 years of
mathematical and philosophical
research to amend that model and come to
today's model.
A model in which the sum of an infinite
number
of terms can indeed converge to a finite
quantity without contradictions.
Now you see the relevance of this problem
to digital signal processing where we are
measuring physical quantities at regular
intervals in time while assuming that
the underlying physical quantity is
actually continuous.
One of the reasons why arriving at this
better
model of reality took over 2,000 years is
that in
the Middle Ages, as you can see from this
picture,
people were concerned with much more
mundane tasks than refining
a mathematical model of reality.
But progress did come in the end and the
two towering figures of the 17th century,
in this sense, are Galileo and Descartes.
Descartes, the inventor of the Cartesian
plane, started by put a name to think.
So if you have a point on the plane like
so, Descartes said, well,
if I use a coordinate system around this
point I can give a name
to this point and I can use algebraic
formulas
to describe geometrical entities and
perform operation on them.
So for instance, a line would map to a
first degree equation.
This allowed Descartes to solve
algebraically, geometric problems that had
baffled Greeks, such as for instance the
trisection of the angle.
Much more importantly for us is the fact
that the Cartesian Plane is the grand
daddy of all vector spaces and you will
see how
useful vector spaces are in the context of
digital signal processing.
Well, Europe in the 17th century was flush
with
money and Europeans had two things on
their minds.
Finding new markets and winning the war of
conquest that came with the appropriation
of new markets.
Calculus, that was invented in those
years, purported to
provide a new answer to both problems, in
the sense that you could use calculus
to find optimal ship routes around the
globe and to find optimal trajectory for
cannonballs.
Galileo, in particular, worked on the
cannonball problem.
And operated by running a series of
experiments in which
the trajectory of balls thrown by a cannon
was experimentally determined.
And then working backwards to derive an
ideal Platonic model of the balls
trajectory.
That is given by this equation where the
initial velocity, expressed as a vector in
the Cartesian plane, is coupled with the
pull of gravity to give a parabolic shape.
So the way science proceeded was by
starting from set of
experimental data points and then work
backwards to find the description
of the underlying phenomenon in the form
of a perfect algebraic equation.
This usually worked very well for
astronomy, which was a main concern
in those days, because the trajectories of
the planets are perfect conic curves.
The invention of calculus and the
availability of models for reality
based on functions of real variables, led
naturally to what we call
continuous time signal processing.
So if you have a function like this which
is
for instance, a temperature function, you
can compute the average,
in continuous time, by taking the integral
of the function
over its support and dividing by the
length of the support.
Without calculus what you would have to do
is take daily measurements say of the
temperature.
And to compute the average you would just
sum this values together and then divide
by the number of days.
Now the question is what is the relation
between these two averages?
What is the error I incur if I use
experimental data rather
than finding the ideal function behind the
data and then computing the integral.
And even if I can do that for certain
signals that appeared to be smooth
and slow like this one, can I do the same
if the signal is fast?
In other words, if I have a set
of measurements for something that appears
to move quite
quickly, do I have any chance even at
recovering the ideal function that lies
underneath the data.
Well it took a long time since the
17th century to answer this question
because one
of the missing pieces of the puzzle was
how to measure this speed of the signal.
The answer came from Joseph Fourier,
the inventor of Fourier analysis.
That showed us how to decompose any
physical phenomenon, any
description of a physical phenomenon, into
a series of sinusoidal components.
A sinusoidal component is like a wave, and
a wave is parameterized by its
frequency, which is really a way of
measuring how fast the wave oscillates.
You can see an example of this sinusoidal
decomposition of
the signal in the spectral analyzer of
your MP3 player.
By splitting a signal into frequency
components you
can see where the energy of the signal is.
And if it is in the high frequencies then
the signal will be moving very fast.
Whereas if it is in the low frequencies it
will be moving very slow.
The next piece of the puzzle that
completes the path from continuous
reality to discreet reality was given by
Nyquist and
Shannon, two researchers at Bell Labs in
the 50s.
Their sampling theorem is really the
bridge that
connects the analog world to the digital
world.
The formula is like so, and it looks
pretty complicated right now but
you will be very familiar with it by the
end of the class.
If you just have a tiny look at it, you
will see that
the formula relates a continuous time
function, the Platonic ideal
we were talking about, to a set of
discrete time measurements.
And this sum really is a weighted sum
where
for each discrete sample we associate a
special shape.
Graphically it looks like so.
If this is our continuous time function,
the ideal function, and we have a set
of measurements that we indicate with
these red
dots, we can reconstruct the original
functions starting
from the samples just by associating what
we
call a sinc function to each of the
points.
So you scale copies of the same function
at each measured interval and
then when you sum them all together you
obtain the original function back.
All you need for this magic trick to
happen is
that the original function is not too
fast, in
the sense that it doesn't contain too many
high frequencies.
We will see this in more detail during the
class.
If we now go back to the beginning of our
overview, you
remember the second fundamental ingredient
in
digital signals is the discrete amplitude.
What that means is that we have two forms
of discretization of an ideal function.
Take for example this
sine wave.
The first discretization happens in time
and we get a discrete set of samples.
And then the second discretization happens
in amplitude, where each sample
can take values only amongst a
predetermined set of possible levels.
The very important consequence of the
discretization independently of the
number of levels is that the set of levels
is countable.
So we can always map the level of a sample
to an integer.
If our data is just a set of integers now,
it
means that its representation is
completely abstract and completely general
purpose.
This has some very important consequences
in three domains.
Storage becomes very easy because any
memory
support that can store integers can store
signals.
And computer memory comes to mind as a
first candidate.
Processing becomes completely independent
on the nature of the signal because
all we need is a processors that can deal
with the integers.
And again, CPUs are general purpose
processors
that can deal with integers very, very
well.
And finally transmission, with digital
signal we
will be able to deploy very effective
ways to combat noise and transmission
errors as we will see in a second.
As far as storage
is concerned, just consider the difference
between attempting to store
an analog signal, which requires a medium
dependent support for
each kind of application, and the task of
storing a
digital signal, which requires just a
piece of computer memory.
In analog storage the medium evolved as
technology evolved.
And for instance when it came to sound we
had wax
cylinders in the beginning, and then you
had vinyl, and then
reel-to-reel tapes, and then compact
cassettes, and so on and so forth.
Each medium was incompatible with its
predecessor
and required specialized hardware to be
reproduced.
Today everything is stored in general
purpose support systems, like memory card
or a hard drive, and is completely
independent on the type of
content that is recorded.
If you consider for instance the evolution
of memory
supports, this is a famous picture from
the internet.
Just one microSD card will contain all the
information
that was contained in countless floppy
discs and CDs from just a few years back.
But what does not change, although the
support changes and the capacity improves
with time, what does not change is the
format
of the data which will remain the same
across media.
When it comes to processing again, the
fact that the representation of the data
is
completely decoupled from the origin of
the data
will allow us to use general purpose
machines.
Here on the left you have three three
analog processing devices.
A thermostat on top with its temperature
sensitive coil,
you have a set of gears that for instance
can be
used to measure movement or time, and a
discrete electronics amplifier.
Each of these devices had to be designed
and
built to process just one type of analog
signal.
Conversely, on the right, you see just a
piece of C code that implements a digital
filter.
Now this filter can be used to process a
temperature signal or a sound signal and
its structure or its implementation will
not change.
Finally, let's consider the problem of
data
transmission which is probably the domain
where
digital signal processing has made the
most difference in our day to day life.
So if you have a communication channel and
you try to send information from
a transmitter to a receiver you are faced
with a fundamental problem of noise.
So let's see what happens inside the
channel.
You have a signal
that will be put into the channel.
The channel will introduce an attenuation.
It will lower the volume of the signal, so
to speak.
But it will also introduce some noise,
indicated here as sigma of t.
And what you will receive at the end is
an attenuated copy of your original
signal, plus noise.
This is just facts of nature that you
cannot escape.
So, if this is your original signal what
you will get at the end is an attenuated
copy scaled by a factor of G, plus noise.
So how do you recover the original
information?
Well, you try to undo the effects
introduced by the
channel but the only thing you can undo is
the attenuation.
So you can try and multiply the received
signal by a gain
factor that is the reciprocal of the
attenuation introduced by the channel.
So if you do that you introduce a gain
here at the receiver and what you get is,
let's start again with the original
signal, attenuated copy,
some noise added and then let's undo the
attenuation.
Well what happens unsurprisingly is that
the gain factor has
also amplified the noise that was
introduced by the channel.
So you get a copy of the signal that is
yes,
of a comparable amplitude to the original
signal, but
in which the noise is much larger as well.
This is typical situation that you get in
second
generation or third generation copies of
say a tape.
Or if you try and do a photocopy of a
photocopy, just
to give you an idea of what happens with
this noise amplification problem.
Now, why is this very important?
This is important because if you have a
very long
cable, so for instance, if you have a
cable that goes from Europe
to the United States, and you try to send
a telephone conversation over there.
What happens is that you have to split the
channel into several
chunks and try to undo the attenuation of
a chunk in sequence.
So you actually put what are called,
repeaters along the
line that regenerated the signal to the
original level every say,
ten kilometers of cable or so.
But unfortunately the cumulative effect of
this chain of receiver, is that some
noise gets introduced at each stage and
gets amplified over and over again.
So for instance, if this is our original
signal which again, gets
attenuated and gets corrupted by noise in
the first segment of the cable.
After amplication you would get this,
which is in that before.
Then this signal is injected into the
second section of the cable.
It gets attenuated.
New noise gets added to it and when
you amplify it you get double the
amplified noise.
And after N sections of the cable you have
N times the amplified noise.
This can lead very quickly to a
complete loss of intelligibility in a
phone conversation.
Let's now consider the problem of
transmitting a digital
signal over the same trans oceanic cable.
Now, a digital signal as we said before,
is composed of samples whose values belong
to a
countable finite set of levels and so
their
values can be mapped to a set of integers.
Now transmitting a set of integers means
that we can encode
these integers in binary format and
therefore we end up transmitting
basically just a sequence of zeros and
ones, binary digits.
We can build an analog signal associating
say the level plus five
volt to the digit zero and minus five volt
to the digit
one and we will have a signal and we will
have a
signal that will oscillate between these
two levels as the digits are transmitted.
What happens on the channel is the same
as before, we will have an attenuation, we
will have the addition of noise and
we will have an amplifier at each repeater
that will try to undo the attenuation.
But on top of it all, we will have what is
called a
threshold operator that will try to
reconstitute
the original signal as best as possible.
Let's see how that works.
If this is what we transmit, say an
alternation of zero and one mapped to
these two
voltage levels, the attenuation and the
noise
will reduce the signal to this state.
The amplification will regenerate the
levels and will amplify the noise.
So, the noise is much larger than before.
But now we can just threshold and say, if
the
signal value is above zero we just output
five volts.
And vice versa, if it's below zero we will
output minus five volts.
So the thresholding operator will
reconstruct a signal like so.
So you can see that at the end of the
first repeater we actually
have an exact copy of the transmitted
signal and not a noise corrupted copy.
The effectiveness of the digital
transmission schemes
can be appreciated by looking at the
evolution
of the throughput, the amount of
information
that can be put on a transatlantic cable.
In 1866,
the first cable was laid down and it had a
capacity of
eight words per minute which corresponded
to approximately five bits per second.
In 1956 when the first digital cable was
laid down on the ocean floor
the capacity all of the sudden sky
rocketed to three mega bits per second.
So, ten to the power of six.
Six order of magnitudes larger than the
analog cable.
And in 2005 when a fiber cable was laid
down another six
order of magnitude were added for a
capacity of 8.4 terabits per second.
Similarly and literally closer to home, we
can look
at the evolution of the throughput for
in-home data transmission.
In the 50s the first voice-band modems
came out of Bell Labs.
Voice-band meaning that they were devices
designed
to operate over a standard telephone
channel.
Their capacity was very low, 1200 bits per
second, and they were analog devices.
With the digital revolution in the 90s
digital modems
started to appear and very quickly reached
basically the ultimate
limit of data transmission over the
voice-band channel which was
56 kilobits per second at the end of the
90s.
The transition to ADSL pushed that limit
up to over 24 megabits per second in 2008.
Now, this evolution is of course partly
due
to improvements in electronics and to
better phone lines.
But fundamentally its success and its
affordability are
due to the use of digital signal
processing.
We can use small yet very powerful and
cheap general purpose processors
to bring the power of error correcting
codes
and data recovery even in small home
consumer devices.
In the next few weeks we will study
signal processing stun, starting from the
ground up.
And by the end of the class will have
enough tricks in our bag to fully
understand how an ADSL modem works.
And so after this very far reaching and
probably rambling introduction, it's time
to go back to basics and we will
see you in module two to discover what
discreet time signals are all about.