1
00:00:00,880 --> 00:00:05,050
My name is Ashish Mahabal and
we'll be continuing with R.

2
00:00:05,050 --> 00:00:07,510
Today we are going to
see more about ggplot.

3
00:00:08,670 --> 00:00:14,550
So let's take a look at the graphic
packages that are available in R and

4
00:00:14,550 --> 00:00:16,360
do a quick comparison of them.

5
00:00:16,360 --> 00:00:21,120
The base graphic has been available right
from the start because it came from S and

6
00:00:21,120 --> 00:00:23,370
Ross Ihaka was instrumental in that.

7
00:00:24,550 --> 00:00:28,170
You can do most of the things
with the base graphics but

8
00:00:28,170 --> 00:00:29,986
it is not easy to modify that.

9
00:00:29,986 --> 00:00:33,960
It can be all low level things but
if you want to do some complicated or

10
00:00:33,960 --> 00:00:36,650
complex plot, then you know,
to really go to a lot of trouble.

11
00:00:38,220 --> 00:00:40,690
After that, in about 2000,
Grid graphics came into the picture.

12
00:00:41,890 --> 00:00:47,370
These are low level but
they do not support statistical plotting.

13
00:00:47,370 --> 00:00:52,980
And that is sort of against the,
our principal because our

14
00:00:52,980 --> 00:00:58,250
is mostly about statistical methodology,
statistical procedures and if you cannot

15
00:00:58,250 --> 00:01:02,880
do a lot of statistical graphics within a
plotting package, that's not very useful.

16
00:01:02,880 --> 00:01:04,950
The Lattice package came
into picture around 2008 and

17
00:01:04,950 --> 00:01:07,390
that is much more powerful.

18
00:01:07,390 --> 00:01:11,460
It is based on grid, it is detailed,
but then it lacks a formal model.

19
00:01:12,730 --> 00:01:18,400
And so,
a bit before that in 2005 ggplot2 that

20
00:01:18,400 --> 00:01:23,630
came on to the scene made with
a very welcome attitude from

21
00:01:23,630 --> 00:01:27,610
everybody because that is where you can
do lots of very interesting things and

22
00:01:27,610 --> 00:01:33,860
all the power of R could be brought to
bear on radius analysis that one can do.

23
00:01:33,860 --> 00:01:37,060
So here, it is a layered structure and

24
00:01:37,060 --> 00:01:39,460
it is based on something
called grammar of graphics.

25
00:01:39,460 --> 00:01:41,910
And we'll be seeing more of that soon.

26
00:01:41,910 --> 00:01:46,020
It is low level and
you can build static plots with it.

27
00:01:46,020 --> 00:01:50,710
The other important thing, is that
extensions to ggplot2 are very easy and

28
00:01:50,710 --> 00:01:54,360
that makes it another welcome addition.

29
00:01:55,720 --> 00:01:57,650
The most important however,

30
00:01:57,650 --> 00:02:02,320
is that the flow of ggplot2 is
like that of your analysis itself.

31
00:02:02,320 --> 00:02:05,320
Because it is layered,
you can think of one thing at a time.

32
00:02:05,320 --> 00:02:06,910
You can start with one piece.

33
00:02:06,910 --> 00:02:08,080
You can build it.

34
00:02:08,080 --> 00:02:09,910
And then on top of another,
on top of that,

35
00:02:09,910 --> 00:02:12,350
you can build another layer,
a third layer and so on.

36
00:02:12,350 --> 00:02:15,660
So, if you want to say, plot points,
you know, plot points and

37
00:02:15,660 --> 00:02:20,120
then you say oh, but what we also
need is to do some smoothing on this.

38
00:02:20,120 --> 00:02:21,610
So you can go ahead and

39
00:02:21,610 --> 00:02:25,160
do the smoothing on that plot as
an additional layer without having to

40
00:02:25,160 --> 00:02:30,050
redo the earlier calculation, or without
having to relay the earlier structure.

41
00:02:30,050 --> 00:02:33,660
On top of that, suppose you decide
that you want error bars plotted

42
00:02:33,660 --> 00:02:35,370
with the smoothing, you can do that too.

43
00:02:35,370 --> 00:02:38,960
So, in a sense, we can have many,
many layers added to a single plot and

44
00:02:38,960 --> 00:02:42,160
build your plot just like you
would build your own analysis.

45
00:02:42,160 --> 00:02:42,739
The flow is the same.

46
00:02:44,460 --> 00:02:47,400
rgobi, ggobi, and rggo that came in 2007,

47
00:02:47,400 --> 00:02:51,930
2008 also use the grammar of graphics,
but they are more interactive.

48
00:02:51,930 --> 00:02:57,050
So you can have an interactive plot and
make changes in that and so on.

49
00:02:57,050 --> 00:03:01,360
The latest kid on the block,
just about one year old, is ggvis.

50
00:03:01,360 --> 00:03:05,000
And that is also an extremely
powerful visualization tool.

51
00:03:05,000 --> 00:03:09,600
It is browser based, so that brings
the game right to your browser.

52
00:03:09,600 --> 00:03:15,280
And because our studio has an elementary
browser, you can also do the browsing and

53
00:03:15,280 --> 00:03:18,966
plot building within our studio itself
which you should be using by now.

54
00:03:18,966 --> 00:03:23,810
So this is also interactive graphics and
we won't be using more of that but

55
00:03:23,810 --> 00:03:27,630
I urge you to explore these other
packages, especially ggvis also.

56
00:03:27,630 --> 00:03:30,920
And you can see more details on
the URL that I have mentioned here.

57
00:03:33,400 --> 00:03:35,190
So, what is the grammar of graphics?

58
00:03:35,190 --> 00:03:39,980
Just like the grammar of a language
has various concepts, and there is

59
00:03:39,980 --> 00:03:43,690
a structure to each sentence, there is
a relationship between different words,

60
00:03:43,690 --> 00:03:46,080
the verbs, adjectives, noun and so on.

61
00:03:46,080 --> 00:03:50,980
So similarly, in the case of grammar
of graphics, it looks at graphics as

62
00:03:50,980 --> 00:03:55,560
a picture built of various elements which
have specific connections with each other.

63
00:03:55,560 --> 00:03:57,482
So, what are the key words here?

64
00:03:57,482 --> 00:04:02,280
We have a data, in case of are it's,
in the form of a data frame,

65
00:04:02,280 --> 00:04:05,310
and the data frame has various columns.

66
00:04:05,310 --> 00:04:10,760
So on these columns, what you do
is you define certain mappings and

67
00:04:10,760 --> 00:04:12,920
you associate geoms with them.

68
00:04:12,920 --> 00:04:15,210
You're geom could be a point,
it could be a line,

69
00:04:15,210 --> 00:04:17,830
it could be some kind of 2D density.

70
00:04:17,830 --> 00:04:23,130
So, given certain columns from your data
frame, you map them using these geoms.

71
00:04:23,130 --> 00:04:27,300
And then additionally you include
some statistical transformation.

72
00:04:27,300 --> 00:04:30,820
And the statistical transformation again
could be something like smoothing or

73
00:04:30,820 --> 00:04:31,680
benign, et cetera.

74
00:04:32,840 --> 00:04:36,780
Most geoms have a default
associated stat with them.

75
00:04:36,780 --> 00:04:40,450
Similarly, most stats have a default
associated genre with them.

76
00:04:40,450 --> 00:04:43,730
But you can do mix and
match with different sets of those,

77
00:04:43,730 --> 00:04:47,010
and we'll see a partial
list of each of them later.

78
00:04:48,040 --> 00:04:52,090
The, another interesting aspect is
that you can do faceting on this data.

79
00:04:52,090 --> 00:04:56,250
By facet, what one means is that
if one has a categorical variable,

80
00:04:56,250 --> 00:05:00,550
a variable which is not continuous and
has some discreet values,

81
00:05:00,550 --> 00:05:06,210
then you can divide your data, or facet
your data on that particular variable.

82
00:05:06,210 --> 00:05:10,833
And ggplot allows you to very
easily plot each variable

83
00:05:10,833 --> 00:05:14,830
separately in a separate sub-plot,
and that can be extremely useful.

84
00:05:14,830 --> 00:05:17,610
We'll be seeing an example of that later.

85
00:05:17,610 --> 00:05:20,710
And then the other two keywords
that get used are scale and

86
00:05:20,710 --> 00:05:24,420
coord where you apply a certain scaling,
you train a scale.

87
00:05:24,420 --> 00:05:26,630
And then you use a coordinate
system with that.

88
00:05:26,630 --> 00:05:30,200
So given these different keywords
in the grammar of graphics,

89
00:05:30,200 --> 00:05:34,230
you can layout in your mind how
your plot should look like and

90
00:05:34,230 --> 00:05:39,516
then there're easily usable techniques,
easily usable keywords within

91
00:05:39,516 --> 00:05:42,570
ggplot that allow you to layout
your layers within the plot.

92
00:05:43,780 --> 00:05:47,850
So to start using ggplot, you would
do that as you do any other package.

93
00:05:47,850 --> 00:05:52,400
So, you'd say install package ggplot2 and
to load it you say library ggplot2.

94
00:05:52,400 --> 00:05:56,020
So, and the URL is up there
where you can get it from.

95
00:05:56,020 --> 00:05:58,518
The data set that we'll be
using is from astronomy.

96
00:05:58,518 --> 00:06:04,480
We have a, a transient survey, the
transient survey, and for the transient

97
00:06:04,480 --> 00:06:09,320
survey, what we do is that we obtain
lightcurves for many, many objects.

98
00:06:09,320 --> 00:06:12,260
In fact, it is using these
lightcurves that we detect whether

99
00:06:12,260 --> 00:06:14,720
a particular object is a transient or not.

100
00:06:14,720 --> 00:06:17,270
A transient being
an object that changes in

101
00:06:17,270 --> 00:06:20,590
brightness in a relatively
short amount of time.

102
00:06:20,590 --> 00:06:22,300
Here we see three examples.

103
00:06:22,300 --> 00:06:24,000
We won't be going into detail here,

104
00:06:24,000 --> 00:06:29,090
but the description of the dataset will
have a little bit more on the website.

105
00:06:29,090 --> 00:06:34,640
So, the data set that we'll use
here is called crts_6class.

106
00:06:34,640 --> 00:06:37,400
That is because this data set
has information about six

107
00:06:37,400 --> 00:06:40,044
different classes and
that is what we'll be using.

108
00:06:40,044 --> 00:06:43,080
There are a little over 1600 rows.

109
00:06:43,080 --> 00:06:45,300
There are 20 useful variables.

110
00:06:45,300 --> 00:06:48,081
And there is one categorical variable,
namely the class or

111
00:06:48,081 --> 00:06:49,200
the type of the object.

112
00:06:50,500 --> 00:06:52,630
This is how a typical light curve looks.

113
00:06:52,630 --> 00:06:54,540
So a light curve is a time series.

114
00:06:54,540 --> 00:06:59,090
On the X axis you have time,
in this case it is a phase time,

115
00:06:59,090 --> 00:07:04,330
that is you use a particular wrap
around at the same period again and

116
00:07:04,330 --> 00:07:06,170
again, so you get a phased light curve.

117
00:07:06,170 --> 00:07:08,570
And on the y-axis is the brightness.

118
00:07:08,570 --> 00:07:12,380
And a couple of parameters that
can be derived from such a light

119
00:07:12,380 --> 00:07:14,960
curve are shown on the plot.

120
00:07:14,960 --> 00:07:17,420
Now, you derive many different parameters.

121
00:07:17,420 --> 00:07:20,230
Mean, median, a skew kurtosis, and so on.

122
00:07:20,230 --> 00:07:24,510
And it is exactly these kinds
of numbers that are available

123
00:07:24,510 --> 00:07:27,470
in the data set that we have.

124
00:07:27,470 --> 00:07:31,560
So, the first ten lines and
only the first few columns are shown here.

125
00:07:31,560 --> 00:07:33,280
Amplitude is the last column shown.

126
00:07:33,280 --> 00:07:38,100
That is the first use for variable that
we can use because position parameters,

127
00:07:38,100 --> 00:07:44,490
we won't use them for trying to
define any particular graphing here.

128
00:07:44,490 --> 00:07:47,920
So, if one wanted to simply take,
say, the amplitude and

129
00:07:47,920 --> 00:07:50,330
standard deviation of all these points and

130
00:07:50,330 --> 00:07:53,608
plot them, in the base graphics it
is a fairly straightforward thing.

131
00:07:53,608 --> 00:07:57,060
All you had to do is say
plot amplitude comma STD,

132
00:07:57,060 --> 00:08:02,590
provided both those variables have
been attached to your username space,

133
00:08:02,590 --> 00:08:06,709
and then you'll just get a plot that shows
amplitude with a standard deviation.

134
00:08:08,170 --> 00:08:10,930
Now ggplot2, has two different techniques.

135
00:08:10,930 --> 00:08:14,040
Qplot, which is the simpler one,
but not as powerful, and

136
00:08:14,040 --> 00:08:18,020
ggplot, which is the more powerful, and
that's the one that you should be using.

137
00:08:18,020 --> 00:08:21,180
But we also should see,
how qplot can be used.

138
00:08:21,180 --> 00:08:25,540
So in case of qplot 2, or which is
essentially been written to do a mapping

139
00:08:25,540 --> 00:08:30,550
to the plot, base graphics, we can simply
say again qplot amplitude was steady and

140
00:08:30,550 --> 00:08:32,530
you'll get the same functionality.

141
00:08:32,530 --> 00:08:36,990
In this case of course
the data=crts_6class has to be attached at

142
00:08:36,990 --> 00:08:37,790
some point before.

143
00:08:37,790 --> 00:08:40,890
Otherwise, you are to
stipulate that as an argument.

144
00:08:40,890 --> 00:08:44,270
And the plot looks slightly different,
but it's exactly the same functionality.

145
00:08:44,270 --> 00:08:46,810
Now when it comes to ggplot,

146
00:08:46,810 --> 00:08:52,440
there, you have to understand that you
had to provide the columns as aesthetics.

147
00:08:52,440 --> 00:08:55,200
And that is what is seen
in the first example here.

148
00:08:55,200 --> 00:08:59,860
We provide to ggplot, a single argument,
the aesthetics and within aesthetics,

149
00:08:59,860 --> 00:09:04,140
which itself is a function, which provide
the column names, Amplitude and STD.

150
00:09:05,180 --> 00:09:08,250
But here, the data equal to crts_6class or

151
00:09:08,250 --> 00:09:11,490
whatever the data frame name is,
you have to provide it.

152
00:09:11,490 --> 00:09:15,280
Unlike Qplot, it is not going to
take it from an earlier invocation.

153
00:09:15,280 --> 00:09:17,110
So within the call, you have to give that.

154
00:09:17,110 --> 00:09:20,950
And the first statement there is
going to just return an error saying,

155
00:09:20,950 --> 00:09:23,320
I don't know what data
you're talking about.

156
00:09:23,320 --> 00:09:28,250
So the next one says that lets use
data of the particular data frame and

157
00:09:28,250 --> 00:09:32,490
then we provide amplitude and
STD as the aesthetics.

158
00:09:32,490 --> 00:09:37,310
Now it goes through, but it still
doesn't do any plotting because that's

159
00:09:37,310 --> 00:09:39,080
what we didn't tell it to do, right?

160
00:09:39,080 --> 00:09:41,840
Because there is no
rendering instruction yet.

161
00:09:41,840 --> 00:09:44,140
So the rendering instruction is separate.

162
00:09:44,140 --> 00:09:47,300
And that is what is shown in the last
statement that you see here.

163
00:09:47,300 --> 00:09:51,220
So you invoke ggplot2, and
you'll notice a plus after that.

164
00:09:51,220 --> 00:09:54,320
That is where the layering is coming in.

165
00:09:54,320 --> 00:09:58,230
You say that I want to use the point geom.

166
00:09:58,230 --> 00:10:01,940
And so the geom point here in this
case need not take any argument.

167
00:10:01,940 --> 00:10:02,820
It can take arguments.

168
00:10:02,820 --> 00:10:04,340
We'll see a little bit about that.

169
00:10:04,340 --> 00:10:07,600
But when you combine the two,
then finally you get a plot.

170
00:10:07,600 --> 00:10:08,910
And the plot looks the same as before.

171
00:10:08,910 --> 00:10:11,220
This is just ranting.

172
00:10:11,220 --> 00:10:15,840
So, what you can also do, and this is
the important part, is that you can assign

173
00:10:15,840 --> 00:10:22,330
the first part that we did of ggplot and
assign it to, say, an object called B.

174
00:10:22,330 --> 00:10:27,560
And that of course, does nothing on the
screen, except for making that object B.

175
00:10:27,560 --> 00:10:31,220
And then now separately,
you can add to that object P,

176
00:10:31,220 --> 00:10:35,190
your geom point with no arguments and
viola, you'll get your plot.

177
00:10:36,720 --> 00:10:38,950
Now, I mentioned faceting earlier.

178
00:10:38,950 --> 00:10:41,050
So, that is something
that you can easily do.

179
00:10:41,050 --> 00:10:42,850
Here, we see an example of that.

180
00:10:42,850 --> 00:10:46,810
Here besides amplitude and SDD,
we give one more argument.

181
00:10:46,810 --> 00:10:49,280
Do the aesthetics function?

182
00:10:49,280 --> 00:10:51,190
That is the column called object.

183
00:10:51,190 --> 00:10:54,260
That is where the class of all
different objects is present.

184
00:10:54,260 --> 00:10:56,500
So we can also name
the documents by the way.

185
00:10:56,500 --> 00:11:01,340
Here we say that amplitude is to be
used as x, and STD to be used as Y.

186
00:11:01,340 --> 00:11:05,750
That allows us to exchange the positions
of the functions quite easily.

187
00:11:05,750 --> 00:11:08,830
And then we say that use shape for
that object.

188
00:11:08,830 --> 00:11:12,530
So what happens now is that each of
the different classes that we have,

189
00:11:12,530 --> 00:11:13,540
come with a different shape.

190
00:11:13,540 --> 00:11:17,290
So it's the same plot, but
now you can see that you've in the,

191
00:11:17,290 --> 00:11:20,507
on the right hand side, you'll see a and
the shapes are seen there.

192
00:11:20,507 --> 00:11:23,990
And in this case the shapes are not
very easily distinguishable, so

193
00:11:23,990 --> 00:11:27,670
instead of shape, we could even say
that oh, let me use colors instead.

194
00:11:27,670 --> 00:11:29,780
And so, if you say color equal to object.

195
00:11:29,780 --> 00:11:32,690
Sure enough, each type of object
now has a different color,

196
00:11:32,690 --> 00:11:35,400
in this case it is much
more much better visible.

197
00:11:37,190 --> 00:11:40,520
Then, once you have done that,
you can think of additional things.

198
00:11:40,520 --> 00:11:44,520
Oh, how does the density of it look
like if I were to smooth the data,

199
00:11:44,520 --> 00:11:46,300
how would that look?

200
00:11:46,300 --> 00:11:49,230
If I want to see as a function of x-axis,

201
00:11:49,230 --> 00:11:54,460
what is the centroid of each of
the different layers and what do I see?

202
00:11:54,460 --> 00:11:56,120
So one can easily do that.

203
00:11:56,120 --> 00:11:59,160
All that one has to do is add yet
another geom here.

204
00:11:59,160 --> 00:12:04,360
If you add a geom smooth, then you do
see how that a function will look like.

205
00:12:04,360 --> 00:12:08,260
You see that on the right-hand most
side where there is an outlier.

206
00:12:08,260 --> 00:12:11,730
There is large sigma there but

207
00:12:11,730 --> 00:12:16,060
in the center of the data you see that
it's a very thin line which tells you that

208
00:12:16,060 --> 00:12:20,030
visually though you see a lot of spread,
really there is not that much spread.

209
00:12:20,030 --> 00:12:23,190
So this is something that becomes
then very easy to see with

210
00:12:23,190 --> 00:12:24,800
these simple parameters.

211
00:12:24,800 --> 00:12:26,960
So here in this case we have
three different layers.

212
00:12:26,960 --> 00:12:29,500
We have used a geom point and geom smooth.

213
00:12:30,600 --> 00:12:34,870
Now we can get rid of the outlier and
we can see that better and

214
00:12:34,870 --> 00:12:38,920
we can do lots of more things
with this kind of data set.

215
00:12:38,920 --> 00:12:43,230
So what I'm going to do now here, we have
gone back to using a single color for

216
00:12:43,230 --> 00:12:44,306
all objects together.

217
00:12:44,306 --> 00:12:48,480
We had earlier seen that one
can have the geom point and

218
00:12:48,480 --> 00:12:50,020
geom smooth with all those points but

219
00:12:50,020 --> 00:12:56,390
what happens if we, now, use the faceting
onto objects, and do the smoothing.

220
00:12:56,390 --> 00:12:59,960
So, in fact, we can see for
each class, just for

221
00:12:59,960 --> 00:13:01,810
that class, how the smoothing looks like.

222
00:13:01,810 --> 00:13:05,300
So, you can immediately start seeing
there the different populations,

223
00:13:05,300 --> 00:13:07,920
maybe even in this very dense cluster.

224
00:13:09,670 --> 00:13:13,954
So, if you have such a dense cluster, what
one could try to do is ungittle a little

225
00:13:13,954 --> 00:13:17,734
better.word g/ them a little bit and
see the spread better and there is

226
00:13:17,734 --> 00:13:22,790
another available for that called so
this allows you to see how that one looks.

227
00:13:25,436 --> 00:13:29,486
Or you can add an alpha parameter to that,
through another layer or

228
00:13:29,486 --> 00:13:32,130
give that as an argument to geom_jitter.

229
00:13:32,130 --> 00:13:33,960
So here we are saying that
the alpha parameter 0.75.

230
00:13:33,960 --> 00:13:37,760
So you see that some of the points
have become more transparent,

231
00:13:37,760 --> 00:13:41,640
that allows you to see,
where the density is better than before.

232
00:13:41,640 --> 00:13:45,400
All that can be are the geoms
like the boxplot in this case,

233
00:13:45,400 --> 00:13:49,010
so you see how boxplots for
each of the objects look.

234
00:13:49,010 --> 00:13:53,299
And again, you can see that command is
being only slightly being modified with

235
00:13:53,299 --> 00:13:57,640
some different geom and you get quite
interesting and straightforward plots.

236
00:13:57,640 --> 00:14:00,830
Or you could look at the geomic density.

237
00:14:00,830 --> 00:14:04,890
And you can see how the density is as
a function of amplitude, so different

238
00:14:04,890 --> 00:14:10,450
classes you can immediately see how they
occupy different parts of the space.

239
00:14:10,450 --> 00:14:13,960
Now if you wanted to really see each
of them in a separate sub-plot,

240
00:14:13,960 --> 00:14:18,050
you can do that using the facet
that I mentioned earlier.

241
00:14:18,050 --> 00:14:21,130
So here we are telling GGPlot to use

242
00:14:21,130 --> 00:14:24,910
object as a faceting parameter
of the categorical variable.

243
00:14:24,910 --> 00:14:28,290
And again, the original command is
the same ggplot, geom point and

244
00:14:28,290 --> 00:14:29,570
just the faceted grid.

245
00:14:29,570 --> 00:14:34,460
And you see that the six types of
objects as seen in a separate subplot.

246
00:14:34,460 --> 00:14:40,080
The important thing,
is that all of them have the same Y scale.

247
00:14:40,080 --> 00:14:44,040
So it is not as if they are being
separately scaled on Y,

248
00:14:44,040 --> 00:14:46,210
because that'll be confusing things here.

249
00:14:46,210 --> 00:14:48,935
We immediately see that the third column,
that of

250
00:14:48,935 --> 00:14:55,210
CVs has a much larger standard deviation
than say, in the first column, the agents.

251
00:14:55,210 --> 00:14:59,170
And that is very important for
it to be done in an automated fashion.

252
00:15:00,520 --> 00:15:04,298
Or what you can then do is that
whatever smoothing that you had done,

253
00:15:04,298 --> 00:15:07,681
you can apply it separately to
these faceted grids also, and

254
00:15:07,681 --> 00:15:11,090
you can see how the smoothed
diversions look on each of them.

255
00:15:12,286 --> 00:15:14,460
W we'll stop and
continue with ggplot2 next time.