1
00:00:00,000 --> 00:00:05,436
Local structure that doesn’t require full table representations
2
00:00:05,452 --> 00:00:09,073
is important in both directed and undirected models.
3
00:00:09,073 --> 00:00:14,868
How do we incorporate local structure into undirected models?
4
00:00:14,868 --> 00:00:22,084
The framework for that is called “log-linear models” for reasons that will be clear in just a moment.
5
00:00:23,084 --> 00:00:24,227
So
6
00:00:26,566 --> 00:00:33,140
Whereas, in the original representation of the unnormalized density
7
00:00:33,140 --> 00:00:39,187
we defined P tilde as the product of factors φi(Di),
8
00:00:39,218 --> 00:00:42,305
each [of] which is potentially a full table.
9
00:00:42,889 --> 00:00:46,032
Now we're going to shift that representation
10
00:00:46,032 --> 00:00:49,975
to something that uses a linear form
11
00:00:50,000 --> 00:00:52,446
(So here's a linear form)
12
00:00:52,446 --> 00:00:54,892
that is subsequently exponentiated,
13
00:00:54,922 --> 00:00:56,970
and that's why it's called log-linear—
14
00:00:56,970 --> 00:00:59,632
because the logarithm is a linear function.
15
00:00:59,632 --> 00:01:02,742
So what is this form over here?
16
00:01:02,742 --> 00:01:06,986
It's a linear function that has these things that are called “coefficients”
17
00:01:10,171 --> 00:01:12,631
and these things that are called “features”.
18
00:01:14,200 --> 00:01:22,108
Features, like factors, each have a scope which is a set of variables on which the feature depends.
19
00:01:24,153 --> 00:01:27,026
But different features can have the same scope.
20
00:01:27,026 --> 00:01:31,176
You can have multiple features all of which are over the same set of variables.
21
00:01:31,176 --> 00:01:36,472
Notice that each feature has just a single parameter wj that multiplies it.
22
00:01:37,657 --> 00:01:40,310
So, what does this give rise to?
23
00:01:40,310 --> 00:01:43,456
I mean if we have a log-linear model,
24
00:01:43,487 --> 00:01:49,465
we can push in the exponent through the summation,
25
00:01:49,465 --> 00:01:58,764
and that gives us something that is a product of exponential functions.
26
00:01:58,795 --> 00:02:03,530
You can think of each of these as effectively a little factor,
27
00:02:03,530 --> 00:02:07,689
but it’s a factor that only has a single parameter wj.
28
00:02:07,689 --> 00:02:11,201
Since this is a little bit abstract, so let’s look at an example.
29
00:02:11,201 --> 00:02:17,694
Specifically lets look at how we might represent a simple table factor as a log linear model.
30
00:02:17,709 --> 00:02:24,290
So here’s a param, here’s a factor φ, over two binary random variables X1 and X2.
31
00:02:24,290 --> 00:02:30,788
And so a full table factor would have four parameters: a00, a01, a10, and a11.
32
00:02:30,788 --> 00:02:34,552
So we can capture this model using a log linear model,
33
00:02:34,552 --> 00:02:37,958
using a set of such of features,
34
00:02:37,958 --> 00:02:41,365
using a set of these guys, which are indicator functions.
35
00:02:41,365 --> 00:02:43,225
So this is an indicator function.
36
00:02:43,225 --> 00:02:48,963
It takes one if X1 is zero and X2 is zero,
37
00:02:48,978 --> 00:02:50,881
and it takes zero otherwise.
38
00:02:51,035 --> 00:02:54,092
So this the general notion of an indicator function.
39
00:02:54,123 --> 00:03:00,315
It looks at the event—or constraint—inside the curly braces,
40
00:03:00,407 --> 00:03:05,230
and it returns a value of 0 or 1, depending on
whether that event is true or not.
41
00:03:05,722 --> 00:03:13,595
And so, if we wanted to represent this factor as a log-linear model,
42
00:03:13,626 --> 00:03:19,920
We can see that we can simply sum up over all the four values of k and ℓ,
43
00:03:19,920 --> 00:03:22,691
which are either 0 or 1, each of them.
44
00:03:22,707 --> 00:03:25,407
So were summing up over all four entries here.
45
00:03:25,407 --> 00:03:32,480
And we have a parameter—or coefficient—wkℓ which multiplies this feature.
46
00:03:33,295 --> 00:03:44,124
And so, we would have a summation of wkℓ:
47
00:03:44,139 --> 00:03:49,224
of w00 only in the case where X1 is zero and X2 is zero.
48
00:03:49,255 --> 00:04:00,483
So we would have exp of negative w00 when X1=0 and X2=0,
49
00:04:01,114 --> 00:04:12,037
and we would have exp of negative w01 when
X1=0 and X2=1, and so on and so forth.
50
00:04:12,390 --> 00:04:15,509
And it’s not difficult to convince ourselves that
51
00:04:15,525 --> 00:04:24,771
if we define wkℓ to be the negative log of the corresponding entries in this table,
52
00:04:24,801 --> 00:04:28,650
then that gives us right back the factor that
we defined to begin with.
53
00:04:28,681 --> 00:04:33,144
So this shows that this is a general representation,
54
00:04:34,005 --> 00:04:37,267
in the sense that we can take any factor
55
00:04:37,898 --> 00:04:40,529
and represent it as a log-linear model
56
00:04:40,544 --> 00:04:47,175
simply by including all of the appropriate features.
57
00:04:48,605 --> 00:04:52,359
But we don’t generally want to do that.
58
00:04:52,359 --> 00:04:55,800
Generally we want a much finer grain set of features.
59
00:04:55,800 --> 00:05:01,197
So let’s look at some of the examples of features that people use in practice.
60
00:05:01,197 --> 00:05:03,838
So here are the features used in a language model.
61
00:05:03,853 --> 00:05:07,934
This is a language model that we that we discussed previously.
62
00:05:07,950 --> 00:05:12,276
And here we have features that relate:
63
00:05:12,292 --> 00:05:15,839
First of all, let’s just remind ourselves [that] we have two sets of variables.
64
00:05:15,839 --> 00:05:23,683
We have the variables Y which represent the annotations for each word
65
00:05:23,698 --> 00:05:29,956
in the sequence corresponding to what category that corresponds to.
66
00:05:29,956 --> 00:05:32,429
So this is a person.
67
00:05:32,429 --> 00:05:34,903
This is the beginning of a person name.
68
00:05:34,936 --> 00:05:37,847
This is the continuation of a person name.
69
00:05:37,847 --> 00:05:39,302
The beginning of a location.
70
00:05:39,302 --> 00:05:42,235
The continuation of a location, and so on.
71
00:05:42,235 --> 00:05:45,694
As well as a bunch of words that are not:
72
00:05:45,694 --> 00:05:48,444
[i.e.,] none of person, location, organization.
73
00:05:48,444 --> 00:05:50,768
And they’re all labeled “other”.
74
00:05:50,783 --> 00:05:54,938
And so the value Y tells us for each word what
category it belongs to,
75
00:05:54,938 --> 00:05:59,375
so that we’re trying to identify people, locations, and
organizations in the sentence.
76
00:05:59,375 --> 00:06:02,146
We have another set of variables X,
77
00:06:03,315 --> 00:06:08,687
which are the actual words in the sentence.
78
00:06:09,225 --> 00:06:12,806
Now we can go ahead and define…
79
00:06:12,806 --> 00:06:15,849
We can use a full table representation that
80
00:06:15,849 --> 00:06:22,729
basically tries to relate each and every Y that has a feature,
81
00:06:22,729 --> 00:06:27,678
that has a full factor that looks at every possible word in the English language;
82
00:06:27,678 --> 00:06:31,674
but those are going to be very, very, expensive,
83
00:06:31,674 --> 00:06:34,070
and a very large number of parameters.
84
00:06:34,117 --> 00:06:40,047
And so we're going to define a feature that looks, for example, at f of
85
00:06:40,047 --> 00:06:45,203
say a particular Yi, which is the label for the i’th word in the sentence,
86
00:06:45,203 --> 00:06:47,652
and Xi, being that i’th word.
87
00:06:47,652 --> 00:06:54,839
And that feature says, for example: Yi equals person.
88
00:06:55,839 --> 00:07:03,750
It’s the indicator function for “Yi = person and Xi is capitalized”.
89
00:07:05,827 --> 00:07:09,694
And so that feature doesn’t look at the individual words.
90
00:07:09,694 --> 00:07:13,192
It just looks at whether that word is capitalized.
91
00:07:13,192 --> 00:07:17,430
Now we have just the single parameter that looks just at capitalization,
92
00:07:17,430 --> 00:07:23,140
and parameterizes how important is capitalization for recognizing that something's a person.
93
00:07:23,153 --> 00:07:26,949
We could also have another feature.
94
00:07:26,949 --> 00:07:28,862
This is an alternative:
95
00:07:28,862 --> 00:07:32,590
This a different feature that can and could be part of the same model
96
00:07:32,590 --> 00:07:38,381
that says: Yi is equal to location,
97
00:07:38,381 --> 00:07:41,061
Or, actually, I was little bit imprecise here—
98
00:07:41,061 --> 00:07:45,464
This might be beginning of person. This might be beginning of location.
99
00:07:45,480 --> 00:07:51,414
And Xi appears in some atlas.
100
00:07:52,137 --> 00:07:55,370
Now there is other things that appear in the atlas than locations,
101
00:07:55,370 --> 00:07:58,648
but if a word appears in the atlas,
102
00:07:58,663 --> 00:08:01,972
there is a much higher probability presumably that it’s actually a location
103
00:08:02,003 --> 00:08:06,199
and so we might have, again, [a] weight for this feature
104
00:08:06,199 --> 00:08:13,611
that indicates that maybe increases the probability in Yi being labeled in this way.
105
00:08:13,611 --> 00:08:19,438
And so you can imagine that constructing a very rich set of features,
106
00:08:19,438 --> 00:08:24,163
all of which look at certain aspects of the word,
107
00:08:24,163 --> 00:08:31,900
and rather than enumerating all the possible words
and giving a parameter to each and one of them.
108
00:08:33,562 --> 00:08:38,073
Let’s look at some other examples of feature-based models.
109
00:08:38,120 --> 00:08:40,913
So this is an example from statistical physics.
110
00:08:40,913 --> 00:08:43,445
It’s called the Ising model.
111
00:08:43,445 --> 00:08:48,778
And the Ising model is something that looks at pairs
of variables.
112
00:08:48,778 --> 00:08:51,773
It’s a pairwise Markov network.
113
00:08:53,127 --> 00:08:55,672
And [it] looks the pairs of adjacent variables,
114
00:08:55,672 --> 00:08:59,495
and basically gives us a coefficient for their products.
115
00:08:59,495 --> 00:09:03,509
So now, this is a case where variables are in the end are binary,
116
00:09:03,509 --> 00:09:07,836
but not in the space {0, 1} but rather
negative one and positive one.
117
00:09:07,836 --> 00:09:10,311
And so now, we have a model that's parametrized
118
00:09:10,311 --> 00:09:13,970
as features that are just the product of the values of the adjacent variables.
119
00:09:14,339 --> 00:09:15,606
Where might this come up?
120
00:09:15,606 --> 00:09:23,165
It comes up in the context, for example, of modeling the spins of electrons in a grid.
121
00:09:23,704 --> 00:09:28,742
So here you have a case where the electrons can rotate
122
00:09:28,742 --> 00:09:31,586
either along one direction or in the other direction
123
00:09:31,586 --> 00:09:40,542
so here is a bunch of the atoms that are marked with a blue arrow.
124
00:09:40,542 --> 00:09:43,059
You have one rotational axis,
125
00:09:43,059 --> 00:09:45,576
and the red arrow[s] are rotating in the opposite direction.
126
00:09:46,084 --> 00:09:52,545
And this basically says we have a term that
127
00:09:52,545 --> 00:09:57,063
[whose] probability distribution over the joint set of spins.
128
00:09:57,063 --> 00:10:01,519
(So this is the joint spins.)
129
00:10:01,950 --> 00:10:08,142
And the model, depends on whether adjacent
atoms have the same spin or opposite spin.
130
00:10:08,142 --> 00:10:12,445
So notice that one times one is the same as negative one times negative one.
131
00:10:12,445 --> 00:10:16,543
So this really just looks at whether they have the same spin
or different spins.
132
00:10:16,543 --> 00:10:22,888
And there is a parameter that looks at, you know, same or
different.
133
00:10:24,088 --> 00:10:26,709
That's what this feature represents.
134
00:10:27,170 --> 00:10:32,514
And, depending on the value of this parameter over here,
135
00:10:32,514 --> 00:10:35,010
if the parameter goes one way,
136
00:10:35,010 --> 00:10:39,599
we're going to favor systems
137
00:10:39,599 --> 00:10:42,509
where the atoms spin in the same direction.
138
00:10:42,509 --> 00:10:48,049
And if it’s going in the opposite direction, you’re going to favor atoms that spin in the different direction.
139
00:10:48,049 --> 00:10:51,268
And those are called ferromagnetic and anti-ferromagnetic.
140
00:10:52,514 --> 00:10:58,536
Furthermore, you can define in these systems the notion of a temperature.
141
00:10:58,813 --> 00:11:04,720
So the temperature here says how strong is this connection.
142
00:11:04,720 --> 00:11:13,126
So notice that as T grows—as the temperature grows—the wij’s get divided by T.
143
00:11:15,695 --> 00:11:19,269
And they all kind of go towards zero,
144
00:11:20,992 --> 00:11:24,812
which means that the strength of the connection between
145
00:11:24,812 --> 00:11:27,896
adjacent atoms, effectively becomes almost moot,
146
00:11:27,896 --> 00:11:30,980
and they become almost decoupled from each other.
147
00:11:30,980 --> 00:11:36,216
On the other hand, as the temperature decreases,
148
00:11:38,201 --> 00:11:43,461
Then the effect of the interaction between the atoms becomes much more significant
149
00:11:43,461 --> 00:11:46,915
and they’re going to impose much stronger constraints on each other.
150
00:11:46,915 --> 00:11:49,428
And this is actually a model of a real physical system.
151
00:11:49,428 --> 00:11:51,941
I mean, this is real temperature, and real atoms, and so on.
152
00:11:51,941 --> 00:11:57,075
And sure enough, if you look at what happens to these models as a function of temperature,
153
00:11:57,075 --> 00:12:00,640
what we see over here is high temperature.
154
00:12:02,280 --> 00:12:04,218
This is high temperature
155
00:12:04,218 --> 00:12:10,033
and you can see that there is a lot of mixing between the two types of spin
156
00:12:10,033 --> 00:12:11,925
and this is low temperature
157
00:12:12,587 --> 00:12:20,486
and you can see that there is much stronger
constraints in this configuration
158
00:12:20,486 --> 00:12:23,002
about the spins of adjacent atoms.
159
00:12:26,955 --> 00:12:33,844
Another kind of feature that's used very much in lots of practical applications
160
00:12:33,844 --> 00:12:37,682
is the notion of a metric, of a metric feature, an M.R.F.
161
00:12:37,682 --> 00:12:39,695
So what's a metric feature?
162
00:12:39,695 --> 00:12:41,800
This is something that comes up, mostly in cases
163
00:12:41,800 --> 00:12:48,432
where you have a bunch of random variables Xi that all take values in some joint label space of V.
164
00:12:48,432 --> 00:12:50,269
So, for example, they might all be binary.
165
00:12:50,269 --> 00:12:52,852
They all might take values one, two, three, four.
166
00:12:52,852 --> 00:12:57,250
And what we'd like to do is
167
00:12:57,250 --> 00:13:03,466
we have Xi and Xj that are connected to each other by an edge.
168
00:13:03,466 --> 00:13:08,213
We want X and Xj to take “similar” values.
169
00:13:08,213 --> 00:13:11,968
So in order to enforce the fact that Xi and Xj should take similar values
170
00:13:11,968 --> 00:13:13,939
we need a notion of similarity.
171
00:13:13,939 --> 00:13:24,110
And we're going to encode that using the distance function µ that takes two values, one for Xi and one for Xj’s,
172
00:13:24,110 --> 00:13:26,265
[that] says how close are they to each other.
173
00:13:26,265 --> 00:13:28,939
So what does the distance function need to be?
174
00:13:28,939 --> 00:13:35,552
Well, the distance function needs to satisfy the standard condition on a distance function or a metric.
175
00:13:35,552 --> 00:13:39,375
So first is reflexivity,
176
00:13:39,375 --> 00:13:43,459
which means that if the two variables take on the same value,
177
00:13:43,459 --> 00:13:45,675
then that distance better be zero.
178
00:13:48,598 --> 00:13:53,736
Oh I forgot to say that this. Sorry, this needs to be a non-negative function.
179
00:13:53,736 --> 00:14:00,733
Symmetry means that the distances are symetrical.
180
00:14:00,733 --> 00:14:06,276
So the distance between two values v1 and v2 are the same as the distance between v2 and v1.
181
00:14:06,276 --> 00:14:12,325
And finally is the triangle inequality, which says that the distance between v1 and v2
182
00:14:12,325 --> 00:14:13,505
(So here is v1)
183
00:14:13,505 --> 00:14:14,685
(Here is v2)
184
00:14:15,485 --> 00:14:21,091
and the distance between v1 and v2 is less than the distance between v1 and v3
185
00:14:21,891 --> 00:14:25,128
and then going to v2. So the standard triangle inequality.
186
00:14:26,513 --> 00:14:32,741
if a distance just satisfies these two conditions, it's called a semi metric.
187
00:14:33,848 --> 00:14:37,415
Otherwise, if it satisfies all three, it's called a metric.
188
00:14:37,846 --> 00:14:41,378
And both are actually used in practical applications.
189
00:14:43,932 --> 00:14:48,556
But how do we take this distance feature and put it in the context of an MRF?
190
00:14:48,556 --> 00:14:55,670
We have a feature that looks at two variables, Xi and Xj.
191
00:14:55,670 --> 00:14:59,476
And that feature is the distance between Xi and Xj.
192
00:14:59,476 --> 00:15:08,675
And now, we put it together by multiplying that with a coefficient, wij,
193
00:15:08,675 --> 00:15:13,015
such that wij has to be greater than zero.
194
00:15:13,015 --> 00:15:16,433
So that we want the metric MRF
195
00:15:17,279 --> 00:15:21,189
[to have] the effect that
196
00:15:21,189 --> 00:15:29,346
the lower the distance, the higher this is,
197
00:15:30,269 --> 00:15:36,438
because of the negative coefficient, which means that higher the probability. Okay?
198
00:15:37,007 --> 00:15:43,156
So, the more pairs you have that are close to each other
199
00:15:43,156 --> 00:15:47,280
and the closer they are to each other the higher
the probability of the overall configuration.
200
00:15:47,280 --> 00:15:49,652
Which is exactly what we wanted to have happen.
201
00:15:53,101 --> 00:15:58,418
So, conversely, if you have values that are far from
each other in the distance metric
202
00:15:58,418 --> 00:16:00,596
the lower the probability in the model.
203
00:16:01,796 --> 00:16:04,555
So, here are some examples of metric MRF’s.
204
00:16:04,832 --> 00:16:07,816
So one: The simplest possible metric MRF
205
00:16:07,816 --> 00:16:11,985
is one that gives [a] distance of zero when the two classes are equal to each other
206
00:16:11,985 --> 00:16:13,877
and [a] distance of one everywhere else.
207
00:16:13,877 --> 00:16:16,468
So now, you know, this is just like a step function.
208
00:16:16,468 --> 00:16:22,583
And, this gives rise to a potential that looks like this.
209
00:16:22,583 --> 00:16:26,579
So we have 0’s on the diagonal.
210
00:16:26,579 --> 00:16:33,621
So we get a bump in the probability when the two adjacent variables take on the same label
211
00:16:33,621 --> 00:16:37,122
and otherwise we get a reduction in the probability.
212
00:16:37,137 --> 00:16:40,692
But it doesn’t matter what particular value they take.
213
00:16:40,692 --> 00:16:44,090
That’s one example of a simple metric.
214
00:16:44,090 --> 00:16:51,995
A somewhat more expressive example might come up when the values V are actually numerical values.
215
00:16:52,026 --> 00:16:58,099
In which case you can look at maybe the difference between the miracle values.
216
00:16:58,099 --> 00:17:00,794
So, vk minus vl.
217
00:17:00,794 --> 00:17:05,103
And you want, and when vk is equal to vl, the distance is zero,
218
00:17:05,103 --> 00:17:14,297
and then you have a linear function that increases the
distance as the distance between vk and vl grows.
219
00:17:14,297 --> 00:17:18,733
So, this is the absolute value of vk minus vl.
220
00:17:20,353 --> 00:17:26,010
A more interesting notion that comes up a lot in
practice is:
221
00:17:26,010 --> 00:17:32,374
we don’t want to penalize arbitrarily things that are far way from each other in label space.
222
00:17:32,374 --> 00:17:37,437
So this is what is called a truncated linear penalty.
223
00:17:38,160 --> 00:17:41,852
And you can see that beyond a certain threshold,
224
00:17:44,067 --> 00:17:48,651
the penalty just becomes constant, so it plateaus.
225
00:17:48,667 --> 00:17:54,046
So that there is a penalty, but it doesn’t keep increasing over as the labels get further from each other
226
00:17:54,046 --> 00:17:59,768
One example where metric MRF’s are used is when we’re doing image segmentation.
227
00:17:59,768 --> 00:18:04,136
And here we tend to favor segmentations where adjacent superpixels…
228
00:18:06,551 --> 00:18:09,064
(These are adjacent superpixels.)
229
00:18:10,402 --> 00:18:12,700
And we want them to take the same class.
230
00:18:18,146 --> 00:18:22,279
And so here we have no penalty when the superpixels take the same class
231
00:18:22,279 --> 00:18:25,199
and we have some penalty when they take different classes.
232
00:18:25,199 --> 00:18:30,949
And this is actually a very common, albeit simple, model for
image segmentation.
233
00:18:32,165 --> 00:18:37,141
Let’s look at a different MRF, also in the context of
computer vision.
234
00:18:37,141 --> 00:18:40,440
This is an MRF that’s used for image denoising.
235
00:18:40,440 --> 00:18:45,192
So here we have a noisy version of a real image that looks like this.
236
00:18:45,208 --> 00:18:50,824
So this is, you can see this kind of, white noise overlayed on top of the image.
237
00:18:50,824 --> 00:18:53,923
And what we’d like to do, is we’d like to get a cleaned-up version of the image.
238
00:18:53,923 --> 00:19:00,799
So here we have, a set of variables, X, that correspond to the noisy pixels.
239
00:19:02,230 --> 00:19:07,736
And we have a set of variables, Y, that corresponds to the cleaned pixels.
240
00:19:07,736 --> 00:19:15,427
And we'd like to have a probabilistic model that relates X and Y.
241
00:19:15,427 --> 00:19:20,525
And what we’re going to do is we’d like, so, intuiti—, I mean,
242
00:19:20,525 --> 00:19:25,488
so you’d like to have two effects on the pixels Y:
243
00:19:25,488 --> 00:19:33,105
First, you'd like Yi to be close to Xi.
244
00:19:33,105 --> 00:19:36,372
But if you just do that, then you're just going to stick with
the original image.
245
00:19:36,372 --> 00:19:41,348
So what is the main constraint that we can employ on the image in order to clean it up
246
00:19:41,348 --> 00:19:46,916
is the fact that adjacent pixels tend to have the same value.
247
00:19:46,916 --> 00:19:52,907
So in this case what we’re going to do is we’re going to model, we’re going to constrain the image
248
00:19:52,907 --> 00:20:00,212
so that we’re going to constrain the Yi’s to try and make Yi close to its neighbors.
249
00:20:03,012 --> 00:20:05,831
And the further away it is, the bigger the penalty.
250
00:20:05,831 --> 00:20:08,219
And that's a metric MRF.
251
00:20:11,526 --> 00:20:17,085
Now we could use just a linear penalty,
252
00:20:17,085 --> 00:20:21,892
but that’s going to be a very fragile model,
253
00:20:21,892 --> 00:20:28,208
because, now obviously the right answer isn't the model
where all pixels are equal to each other
254
00:20:28,208 --> 00:20:33,874
in their actual intensity value because that would be just a single, you know, grayish-looking image.
255
00:20:33,874 --> 00:20:39,690
So what you like is that you would like to let one pixel depart from its adjacent pixel
256
00:20:39,690 --> 00:20:44,098
if it’s getting close in a different direction either by its own observation or by other adjacent pixels.
257
00:20:44,098 --> 00:20:49,482
And so the right model to use here is actually the truncated linear model
258
00:20:49,482 --> 00:20:52,174
and that one is [the] one that’s commonly used
259
00:20:52,174 --> 00:20:54,970
and is very successful for doing image denoising.
260
00:20:55,432 --> 00:21:02,063
Interesting, almost exactly the same idea is used in the context of stereo reconstruction.
261
00:21:02,063 --> 00:21:06,264
There, the values that you’d like to infer, the Yi’s,
262
00:21:06,264 --> 00:21:11,557
are the depth disparity for a given pixel in the image—how deep it is.
263
00:21:11,557 --> 00:21:13,957
And here also we have spacial continuity.
264
00:21:13,957 --> 00:21:19,388
We like the depth of one pixel to be close to the depth of an adjacent pixel.
265
00:21:19,388 --> 00:21:22,497
But once again we don’t want to enforce this too strongly
266
00:21:22,497 --> 00:21:25,159
because you do have depth disparity in the image
267
00:21:25,159 --> 00:21:27,820
and so eventually you'd like things to be allowed to break away from each other.
268
00:21:27,820 --> 00:21:33,320
And so once again, one typically uses some kind of truncated linear model
269
00:21:33,320 --> 00:21:36,239
for doing this stereo construction,
270
00:21:36,239 --> 00:21:38,091
often augmented by other little tricks.
271
00:21:38,091 --> 00:21:45,143
So, for example, here we have the actual pixel appearance,
272
00:21:45,143 --> 00:21:47,867
for example, the color and texture.
273
00:21:47,867 --> 00:21:50,392
And if the color and texture are very similar to each other,
274
00:21:50,407 --> 00:21:54,861
you might want to have the stronger constraint on similarity.
275
00:21:54,861 --> 00:22:00,257
Versus: if the color and texture of the adjacent pixels are
very different from each other,
276
00:22:00,257 --> 00:22:02,868
they may be more likely to belong to different objects
277
00:22:02,868 --> 00:22:07,580
and you don’t want to enforce quite as strong of a similarity constraint.