1
00:00:08,607 --> 00:00:11,302
Hi everyone, welcome to today's tutorial,

2
00:00:11,302 --> 00:00:14,420
where we will be looking
at visualization in pandas.

3
00:00:16,230 --> 00:00:20,440
pandas uses matplotlib under the hood, and
provides some convenient functions for

4
00:00:20,440 --> 00:00:21,310
visualizing data.

5
00:00:22,700 --> 00:00:25,540
Before we dive into
visualization in pandas,

6
00:00:25,540 --> 00:00:27,710
lets take a look at
the matplotlib's style package.

7
00:00:29,230 --> 00:00:31,880
Matplotlib comes with a number
of predefined styles,

8
00:00:31,880 --> 00:00:34,980
which we can choose from,
to change the default look of our plots.

9
00:00:36,440 --> 00:00:39,240
Because pandas is using
matplotlib under the hood,

10
00:00:39,240 --> 00:00:41,950
this will change the default style
of our pandas graphs as well.

11
00:00:43,770 --> 00:00:48,910
We can use plt.style.available to
see the predefined styles provided.

12
00:00:52,640 --> 00:00:56,459
Let's use the seaborn-colorblind style,
which will change the default colors of

13
00:00:56,459 --> 00:01:00,074
our plots to use a color palette that is
more color vision deficiency friendly.

14
00:01:03,615 --> 00:01:08,199
Now, where the built-in visualization
of pandas really shines is in

15
00:01:08,199 --> 00:01:11,385
helping with fast and
easy plotting of series and

16
00:01:11,385 --> 00:01:14,440
DataFrames that can help
us explore the data.

17
00:01:14,440 --> 00:01:16,020
Let's make a DataFrame.

18
00:01:16,020 --> 00:01:17,520
First, we'll set the seed for

19
00:01:17,520 --> 00:01:20,820
the random number generator,
which will allow us to reproduce the data.

20
00:01:22,360 --> 00:01:25,750
Next, let's add three columns
of random time series data.

21
00:01:27,460 --> 00:01:31,610
We can generate the random data by
cumulatively summing up random numbers.

22
00:01:32,765 --> 00:01:36,992
numpi has a great function for this called
cumsum, which cumulatively sums an array

23
00:01:39,236 --> 00:01:42,520
Let's do this for
three columns, A, B and C.

24
00:01:42,520 --> 00:01:49,055
And also offset the B column by + 20 and
the C column by- 20.

25
00:01:49,055 --> 00:01:53,921
Using date_range, we can set
the index to be everyday in 2017.

26
00:01:56,647 --> 00:02:01,204
Now let's see what this data looks like a
little more visually to plot this data we

27
00:02:01,204 --> 00:02:03,565
can use the plot method on the DataFrame.

28
00:02:04,890 --> 00:02:10,102
The plot method on series and DataFrame
is just a simple wrapper around plt.plot.

29
00:02:10,102 --> 00:02:11,885
So when we call df.plot,

30
00:02:11,885 --> 00:02:16,439
we get a line graph of all the columns
in the data frame with labels.

31
00:02:17,470 --> 00:02:20,600
Notice how the colors are slightly
different from the default matplotlib

32
00:02:20,600 --> 00:02:22,710
colors because of the style we used.

33
00:02:24,600 --> 00:02:27,550
Also, notice this cool
Jupyter Notebook trick.

34
00:02:27,550 --> 00:02:30,207
If you add a semicolon to
the end of the plotting call,

35
00:02:30,207 --> 00:02:32,108
this will suppress unwanted output.

36
00:02:32,108 --> 00:02:34,627
And this works with regular matplotlib,
too.

37
00:02:37,668 --> 00:02:42,120
DataFrame.plot allows us to plot
a number of different kinds of plots.

38
00:02:42,120 --> 00:02:45,750
We can select which plot we want to use
by pressing it into the kind parameter.

39
00:02:48,020 --> 00:02:52,190
Let's try and create a scatter plot
that takes the A and B columns of df.

40
00:02:53,270 --> 00:02:56,839
We pass in scatter to the kind
parameter to change the plot type.

41
00:02:59,602 --> 00:03:02,890
This is what we'd get if we
called plt.plot on the A and

42
00:03:02,890 --> 00:03:05,550
B columns with the point marker parameter.

43
00:03:07,180 --> 00:03:09,980
Passing scatter into
the kind keyword argument

44
00:03:09,980 --> 00:03:11,870
changed the plot to a scatterplot.

45
00:03:13,950 --> 00:03:17,075
You can also choose the plot
kind by using the methods

46
00:03:17,075 --> 00:03:22,270
dataframe.plot.kind instead of
providing the kind keyword argument.

47
00:03:23,860 --> 00:03:27,370
Now let's try this and
create something a little more complex.

48
00:03:27,370 --> 00:03:32,237
This time we want to make a scatterplot
with points varying in color and size.

49
00:03:32,237 --> 00:03:38,070
We'll use df.plot.scatter,
pass in columns A and C.

50
00:03:39,230 --> 00:03:43,960
And set the color C and size S to
change based on the value of column B.

51
00:03:45,920 --> 00:03:48,890
Finally, we can choose
the color palette used

52
00:03:48,890 --> 00:03:51,310
by passing a string into
the parameter color map.

53
00:03:52,310 --> 00:03:55,540
Here, I'll use viridis,
which is particularly pleasing to the eye.

54
00:03:59,130 --> 00:04:03,109
Here we can see the A and
C columns plotted against one another with

55
00:04:03,109 --> 00:04:06,955
the size and color changing based
on the values of the B column.

56
00:04:06,955 --> 00:04:13,025
Because df.plot.scatter returns
a matplotlib.axes._subplot, we can perform

57
00:04:13,025 --> 00:04:18,520
modification on this object just like
objects returned by matplotlib plots.

58
00:04:20,510 --> 00:04:24,110
For example, let's see what it looks
like when we change the aspect ratio of

59
00:04:24,110 --> 00:04:26,131
the AxesSubplot to equal.

60
00:04:31,480 --> 00:04:36,250
Setting the aspect ratio to equal allows
the viewer to easily see that the range of

61
00:04:36,250 --> 00:04:38,579
series A is much smaller than series C.

62
00:04:41,320 --> 00:04:49,528
Using pandas, we can also easily
do box plots, Histograms,

63
00:04:53,933 --> 00:05:00,690
And kernel density estimate plots.
Which are useful for 
visualizing an estimate of a variable's

64
00:05:00,690 --> 00:05:01,649
probability density function.

65
00:05:01,649 --> 00:05:05,213
Kernel density estimation plots come in
handy in data science application

66
00:05:07,395 --> 00:05:11,481
where you want to derive a smooth continuous
function from a given sample.

67
00:05:12,481 --> 00:05:14,978
Pandas also has plotting tools that help
with visualizing large amounts of data or

68
00:05:15,963 --> 00:05:17,108
high dimensional data.

69
00:05:21,108 --> 00:05:22,380
Let's explore a couple of these tools
by loading in the iris flower data set.

70
00:05:24,140 --> 00:05:27,940

71
00:05:32,701 --> 00:05:36,012
The iris data set is a classic
multivariate data set,

72
00:05:36,012 --> 00:05:38,208
which includes the sepal length,
sepal width, petal length, and
petal width 

73
00:05:38,208 --> 00:05:40,710
for hundreds of samples
of three species of the iris flower.

74
00:05:41,400 --> 00:05:45,760
pandas has a plotting tool that allows
us to create a scatter matrix from a DataFrame.

75
00:05:49,760 --> 00:05:51,270
A scatter matrix is a way of
comparing each column in a DataFrame

76
00:05:51,380 --> 00:05:52,730
to every other column
in a pairwise fashion.

77
00:05:52,730 --> 00:05:56,860
Let's use it to visualize
the iris dataframe and

78
00:05:56,930 --> 00:05:58,930
see what insights we
can gain from our data.

79
00:06:01,560 --> 00:06:03,330
The scatter matrix creates scatter plots
between the different variables and

80
00:06:03,330 --> 00:06:04,630
histograms along the diagonals.

81
00:06:08,630 --> 00:06:10,370
This allows us to quickly see some of
the more obvious patterns in the data set.

82
00:06:11,390 --> 00:06:14,730
Looking at the scatter plots, it appears
that there are some distinct groupings of

83
00:06:16,960 --> 00:06:20,635
the points, which might indicate that
there could be some clustering going on.

84
00:06:20,635 --> 00:06:23,830
And while this looks impressive and

85
00:06:25,430 --> 00:06:26,790
is certainly useful, realize that in
the last two weeks you have gained all of

86
00:06:26,790 --> 00:06:30,830
the skills to manually build these
kinds of plots if you want to.

87
00:06:30,830 --> 00:06:33,740
This is nothing more than a four
by four grid of subplots,

88
00:06:35,060 --> 00:06:38,170
with some plots histograms and
the others scatterplots.

89
00:06:38,170 --> 00:06:41,089
Let's look at one more plotting tool
in pandas which will help us visualize

90
00:06:42,990 --> 00:06:46,460
multivariate data.

91
00:06:46,460 --> 00:06:47,430
pandas includes a plotting tool for
creating parallel coordinates plots.

92
00:06:49,000 --> 00:06:52,260
Parallel coordinate plots are a common
way of visualizing high dimensional multivariate data.

93
00:06:53,520 --> 00:06:58,075
Each variable in the data set
corresponds to an equally

94
00:06:58,075 --> 00:06:59,364
spaced parallel vertical line.

95
00:06:59,364 --> 00:07:02,726
The values of each variable are then
connected by lines between for

96
00:07:02,726 --> 00:07:04,780
each individual observation.

97
00:07:06,320 --> 00:07:10,760

98
00:07:10,760 --> 00:07:12,270
Coloring the lines by class,
in this example the species of flower,

99
00:07:14,540 --> 00:07:16,580
allows the viewer to more easily
see any patterns or clustering.

100
00:07:18,580 --> 00:07:22,030
For instance, looking at our iris data
set, we can see that the petal length and

101
00:07:23,710 --> 00:07:28,160
petal width are two variables that split
the different species fairly clearly.

102
00:07:28,160 --> 00:07:33,070
With iris virginica having the longest and
widest petals.

103
00:07:33,070 --> 00:07:36,970
And iris setosa having the shortest and
narrowest petals.

104
00:07:36,970 --> 00:07:39,070
Well that's it for
visualization in pandas.

105
00:07:40,230 --> 00:07:43,820
In the next video, we'll look at Seaborn,
a powerful package for statistical plotting.