1
00:00:06,519 --> 00:00:10,330
In the examples we've used so far for classification.

2
00:00:10,330 --> 00:00:13,974
We've primarily focused on binary classification,

3
00:00:13,974 --> 00:00:16,839
where the target value to be predicted was

4
00:00:16,839 --> 00:00:21,515
a binary value that was either positive or negative class.

5
00:00:21,515 --> 00:00:28,030
In a lot of real world data sets the target value to be predicted is actually a category.

6
00:00:28,030 --> 00:00:30,039
So for example, in our fruit dataset there were

7
00:00:30,039 --> 00:00:35,420
four different categories of fruit to be predicted, not just two.

8
00:00:35,420 --> 00:00:41,315
So, how do we deal with this multiclass classification situation with scikit-learn?

9
00:00:41,315 --> 00:00:45,340
Well, fortunately scikit-learn makes it very easy to

10
00:00:45,340 --> 00:00:50,869
learn multiclass classification models.

11
00:00:50,869 --> 00:00:53,750
Essentially, it does this by converting

12
00:00:53,750 --> 00:00:56,439
a multiclass classification problem into a series

13
00:00:56,439 --> 00:00:59,454
of binary problems. What do I mean by that?

14
00:00:59,454 --> 00:01:03,009
Well, essentially when you pass in

15
00:01:03,009 --> 00:01:07,734
a dataset that has a categorical variable for the target value,

16
00:01:07,734 --> 00:01:14,995
scikit-learn detects this automatically and then for each class to be predicted.

17
00:01:14,995 --> 00:01:18,640
Scikit-learn creates one binary classifier

18
00:01:18,640 --> 00:01:22,849
that predicts that class against all the other classes.

19
00:01:22,849 --> 00:01:27,250
So for example, in the fruit dataset there are four categories of fruit.

20
00:01:27,250 --> 00:01:34,415
So scikit-learn learns four different binary classifiers.

21
00:01:34,415 --> 00:01:37,180
To predict a new data instance,

22
00:01:37,180 --> 00:01:39,790
what it then does is,

23
00:01:39,790 --> 00:01:42,575
takes that data instance to be predicted,

24
00:01:42,575 --> 00:01:44,484
whose labels to be predict,

25
00:01:44,484 --> 00:01:48,444
and runs it against each of the binary classifiers in turn,

26
00:01:48,444 --> 00:01:53,620
and the classifier that has the highest score is the one that,

27
00:01:53,620 --> 00:01:55,329
whose class it uses,

28
00:01:55,329 --> 00:01:57,245
as the prediction value.

29
00:01:57,245 --> 00:01:59,109
So, let's look at a specific example of

30
00:01:59,109 --> 00:02:02,454
multiclass classification with this fruit dataset.

31
00:02:02,454 --> 00:02:06,670
Here, we simply pass in the normal dataset that

32
00:02:06,670 --> 00:02:12,224
has the value from one to four as the category of fruit to be predicted.

33
00:02:12,224 --> 00:02:14,620
And we fit it exactly the same way that we would

34
00:02:14,620 --> 00:02:21,485
fit the model as if it were a binary problem.

35
00:02:21,485 --> 00:02:24,310
And in general, if we're just, you know, fitting,

36
00:02:24,310 --> 00:02:27,729
and then predicting, all of this would be completely transparent.

37
00:02:27,729 --> 00:02:32,664
Scikit-learn would simply do the right thing and it would learn multiple classes,

38
00:02:32,664 --> 00:02:34,300
and it would predict multiple classes,

39
00:02:34,300 --> 00:02:36,335
and we wouldn't really have to do much else.

40
00:02:36,335 --> 00:02:42,800
However, we can get access to what's happening under the hood as it were,

41
00:02:42,800 --> 00:02:46,030
if we look at the coefficients and the intercepts of

42
00:02:46,030 --> 00:02:51,754
the linear models that result from fitting to the training data.

43
00:02:51,754 --> 00:02:53,979
And this is what this example shows.

44
00:02:53,979 --> 00:02:55,914
So, what we're doing here is fitting

45
00:02:55,914 --> 00:03:02,060
a linear support vector machine to the fruit training data.

46
00:03:02,060 --> 00:03:04,585
And if we look at the coefficient values,

47
00:03:04,585 --> 00:03:12,199
we'll see that instead of just one pair of coefficients for a single linear model,

48
00:03:12,199 --> 00:03:15,129
a classifier, we actually get four values.

49
00:03:15,129 --> 00:03:22,835
And these values correspond to the four classes of fruit in the training set.

50
00:03:22,835 --> 00:03:26,919
And so, what scikit-learn has done here is it's created four binary classifiers,

51
00:03:26,919 --> 00:03:29,020
one for each class.

52
00:03:29,020 --> 00:03:33,580
And so, you can see there are four pairs of coefficients here and

53
00:03:33,580 --> 00:03:38,784
there are also four intercept values.

54
00:03:38,784 --> 00:03:44,080
So, in this case the first pair of coefficients

55
00:03:44,080 --> 00:03:51,414
corresponds to a classifier that classifies apples versus the rest of the fruit,

56
00:03:51,414 --> 00:03:58,594
and so, these pair of coefficients and this intercept define a straight line.

57
00:03:58,594 --> 00:04:03,805
In this case the apples in this visual are the red points, and so,

58
00:04:03,805 --> 00:04:08,259
the coefficients of the apple model define a decision boundary,

59
00:04:08,259 --> 00:04:09,500
a linear decision boundary,

60
00:04:09,500 --> 00:04:14,405
that's marked by this red line here.

61
00:04:14,405 --> 00:04:17,870
And if you plot it out you'll see that indeed it has

62
00:04:17,870 --> 00:04:21,238
an intercept of negative three, and, you know,

63
00:04:21,238 --> 00:04:25,850
you can actually compute for any data instance using

64
00:04:25,850 --> 00:04:31,779
this linear formula with what will predict either apple or not an apple.

65
00:04:31,779 --> 00:04:34,490
So, if we take a specific example,

66
00:04:34,490 --> 00:04:37,564
something that has a height of two

67
00:04:37,564 --> 00:04:42,350
and a width of six so some quick point here, for example.

68
00:04:42,350 --> 00:04:43,714
This is a linear classifier,

69
00:04:43,714 --> 00:04:46,430
so we can take the coefficients.

70
00:04:46,430 --> 00:04:48,259
This first pair of coefficients here,

71
00:04:48,259 --> 00:04:51,379
and this first intercept value here,

72
00:04:51,379 --> 00:04:58,023
and these form a linear classifier for apples versus not apples.

73
00:04:58,023 --> 00:05:02,165
So we can take the height feature,

74
00:05:02,165 --> 00:05:06,728
multiply it by the first coefficient,

75
00:05:06,728 --> 00:05:08,208
take the width feature,

76
00:05:08,208 --> 00:05:11,735
multiply it by the second coefficient,

77
00:05:11,735 --> 00:05:15,019
and then add in the third intercept feature.

78
00:05:15,019 --> 00:05:18,225
This third biased term.

79
00:05:18,225 --> 00:05:20,884
And so, to predict whether this object that

80
00:05:20,884 --> 00:05:23,480
has height of two and width of six is an apple,

81
00:05:23,480 --> 00:05:26,435
we simply compute this linear formula.

82
00:05:26,435 --> 00:05:31,689
It turns out that the value is positive .59.

83
00:05:31,689 --> 00:05:35,449
And so, because that value is greater than or equal to zero that

84
00:05:35,449 --> 00:05:39,500
indicates that the model is predicting that the object is indeed an apple.

85
00:05:39,500 --> 00:05:42,500
And that makes sense because it's on this side of

86
00:05:42,500 --> 00:05:48,290
the apple versus not apple binary classifier decision boundary.

87
00:05:48,290 --> 00:05:50,154
Similarly, we can take another object,

88
00:05:50,154 --> 00:05:54,314
say one who has a height of two and the width of two.

89
00:05:54,314 --> 00:05:57,384
So, in this part of the space over here.

90
00:05:57,384 --> 00:06:04,314
And we can plug in those two feature values into the same apple classifier.

91
00:06:04,314 --> 00:06:08,845
And when we do that it turns out that the prediction value

92
00:06:08,845 --> 00:06:14,029
for the linear model in that case is negative 2.3 and that's less than zero,

93
00:06:14,029 --> 00:06:17,699
which is on this side of the decision boundary.

94
00:06:17,699 --> 00:06:20,839
And so, the linear model for apple versus not apple

95
00:06:20,839 --> 00:06:25,639
here is predicting that this object is not an apple.

96
00:06:25,639 --> 00:06:29,600
And so, again, when scikit-learn has to predict

97
00:06:29,600 --> 00:06:34,475
the class of a new object with potentially multiple classes,

98
00:06:34,475 --> 00:06:41,147
it will go through each of these binary classifiers in turn and it will

99
00:06:41,147 --> 00:06:49,560
predict the class whose classifier has the highest score for that instance.