1
00:00:01,240 --> 00:00:04,030
My name is Ashish Mahabal and
we will be continuing with R.

2
00:00:05,990 --> 00:00:10,120
Today what we are going to do is take
a look at packages, how they're installed,

3
00:00:10,120 --> 00:00:15,040
and also following so of the things that
we see in the best programming practices.

4
00:00:15,040 --> 00:00:21,240
What we are going to do is take a look
at the debugging, profiling et cetera.

5
00:00:21,240 --> 00:00:26,700
So, when it comes to packages, it's
pretty straightforward to install them.

6
00:00:26,700 --> 00:00:30,270
What you can do is
R CMD INSTALL packagename, and

7
00:00:30,270 --> 00:00:34,200
that will go ahead and
install the package.

8
00:00:34,200 --> 00:00:35,279
And if you want to,

9
00:00:35,279 --> 00:00:40,407
you can do a check package name also that
will run the QA that will run the QA so

10
00:00:40,407 --> 00:00:45,480
you would know how Whether the package
name has been installed properly.

11
00:00:45,480 --> 00:00:48,410
And then if you are going
to build a package, a,

12
00:00:48,410 --> 00:00:52,500
a useful thing to know is how to, how to
do that using R CMD build packagename.

13
00:00:52,500 --> 00:00:57,510
And all these instructions will also be on
the sister pages that you'll later on see.

14
00:00:57,510 --> 00:01:01,900
So, when you give the command library,

15
00:01:01,900 --> 00:01:04,280
then that tells you which
packages are already in store.

16
00:01:04,280 --> 00:01:06,800
Of course that's a useful thing to know,
because there are,

17
00:01:06,800 --> 00:01:10,750
as we have seen, more than close to
6,000 packages that are there in R.

18
00:01:11,890 --> 00:01:14,410
And then when you run
library package name,

19
00:01:14,410 --> 00:01:17,590
it lowers the specific package for you.

20
00:01:17,590 --> 00:01:22,190
Here the example that you see on the
screen is about Learn Bayes package about

21
00:01:22,190 --> 00:01:24,450
Bayesian Computation using R.

22
00:01:24,450 --> 00:01:25,970
So when you say library Learn Bayes,

23
00:01:25,970 --> 00:01:30,690
that is the package that's going to
be installed for you loaded for you.

24
00:01:30,690 --> 00:01:33,610
And then when you run the search
command that tells you what

25
00:01:33,610 --> 00:01:34,940
are the packages that are loaded.

26
00:01:34,940 --> 00:01:37,720
So, these are really basic
commands that you get to

27
00:01:37,720 --> 00:01:41,020
know easily once you've used a few times.

28
00:01:41,020 --> 00:01:44,170
So what we are going to do is
we are going to see how one

29
00:01:44,170 --> 00:01:47,020
can handle sets using these now.

30
00:01:47,020 --> 00:01:49,440
There are of course more
automated methods than this, but

31
00:01:49,440 --> 00:01:51,940
it is good to know these
basic methods as well.

32
00:01:53,280 --> 00:01:54,750
So, let's see.

33
00:01:54,750 --> 00:01:57,500
Let's try to use a data
set called achievement.

34
00:01:57,500 --> 00:02:00,380
The way you do it is simply
say data achievement, and

35
00:02:00,380 --> 00:02:01,800
that should load it for you.

36
00:02:01,800 --> 00:02:06,720
And, of course, if you tried to do that,
and if you have not loaded the package in

37
00:02:06,720 --> 00:02:12,510
which case you're going to be told that
it does no where achievement datas is.

38
00:02:12,510 --> 00:02:15,800
And then if you say help but you meant
of course that still won't tell you

39
00:02:15,800 --> 00:02:19,130
much about it because,
again the set is not loading.

40
00:02:19,130 --> 00:02:23,270
Now in this case, if you know that
the data set comes from the package called

41
00:02:23,270 --> 00:02:27,060
learn base, you can quickly check
whether that package has been loaded for

42
00:02:27,060 --> 00:02:30,030
you by giving the search
command as you can see here.

43
00:02:30,030 --> 00:02:34,590
And output of search tells you immediately
that it is in fact not loaded.

44
00:02:34,590 --> 00:02:38,430
So you can go ahead and say I'd like to
relearn base and it will load it for you.

45
00:02:38,430 --> 00:02:40,290
And if you read in the search command,

46
00:02:40,290 --> 00:02:44,990
you will see that on number two you
can see the learn base package there.

47
00:02:44,990 --> 00:02:48,710
So once that is there,
then if you say help achievements,

48
00:02:48,710 --> 00:02:53,420
you're going to see a little bit about
what that particular data set includes.

49
00:02:53,420 --> 00:02:59,470
In this case, it is about a set of
children in Austria, their achievements.

50
00:02:59,470 --> 00:03:01,420
There are 109 observations.

51
00:03:01,420 --> 00:03:04,880
One of the columns is the age
of the children in months,

52
00:03:04,880 --> 00:03:06,940
another column is their IQ.

53
00:03:06,940 --> 00:03:10,170
So, if you wanted to for
instance go ahead blot them,

54
00:03:10,170 --> 00:03:14,270
you could simply say,
plot the age versus the IQ.

55
00:03:16,390 --> 00:03:20,820
And, so one thing that we're going
to see following what we did in

56
00:03:20,820 --> 00:03:26,470
the best programming practices is that
when you run a program, it is important to

57
00:03:26,470 --> 00:03:31,740
save as much time as possible by comparing
different methods, by bench marking it.

58
00:03:31,740 --> 00:03:34,290
So rather than trying
to optimize a program,

59
00:03:34,290 --> 00:03:36,190
what we want to do is then bench mark it.

60
00:03:36,190 --> 00:03:39,010
And the way to do it is through profiling.

61
00:03:39,010 --> 00:03:41,800
So proc.time is
an interesting command that

62
00:03:41,800 --> 00:03:44,030
tells you about just the current time.

63
00:03:44,030 --> 00:03:48,060
So a way to do that would be simply
run it at the start of the program.

64
00:03:48,060 --> 00:03:50,500
The new program, and then out again.

65
00:03:50,500 --> 00:03:53,530
And the difference between
the two times will tell you

66
00:03:53,530 --> 00:03:55,520
how much time has elapsed in the program.

67
00:03:55,520 --> 00:03:57,980
So that is a simple way to do things, but

68
00:03:57,980 --> 00:04:01,360
of course there are more interesting
things that you can use to,

69
00:04:01,360 --> 00:04:05,080
the system of time is required
to evaluate a single expression.

70
00:04:05,080 --> 00:04:07,650
You can use it to evaluate
a single expression.

71
00:04:07,650 --> 00:04:12,570
But then,
using the Rprof is the way to go if you

72
00:04:12,570 --> 00:04:18,170
want to do more complex investigation
into where your program is spending time.

73
00:04:18,170 --> 00:04:21,020
So, Rprof takes a file
name as an argument.

74
00:04:21,020 --> 00:04:23,910
And that, in that file name,
in that file it is going to

75
00:04:23,910 --> 00:04:29,240
put all kinds of information about but
processes are being run, and

76
00:04:29,240 --> 00:04:34,780
when it is time to stop it, you give the
null argument to rprof and run it again.

77
00:04:34,780 --> 00:04:39,970
So, the file name then will contain all
of the things that R has done in between.

78
00:04:39,970 --> 00:04:42,320
But if you look at the file itself,

79
00:04:42,320 --> 00:04:46,660
then you'll find that there's not too
much you can make out of that file.

80
00:04:46,660 --> 00:04:50,860
So, what then you have to do is you
have to run the summaryRprof command on

81
00:04:50,860 --> 00:04:51,860
that filename.

82
00:04:51,860 --> 00:04:56,690
And that is what will summarize for you
what has been going on behind the scenes.

83
00:04:56,690 --> 00:04:59,160
So, here is the plot
that I mentioned earlier.

84
00:04:59,160 --> 00:05:01,630
If you simply try to plot age versus IQ,

85
00:05:01,630 --> 00:05:04,430
of course those belong to
a different namespace and

86
00:05:04,430 --> 00:05:08,810
you'll find that you simply get an error
that it, R doesn't know about agent IQ.

87
00:05:08,810 --> 00:05:12,100
So you could do something
like attach a [INAUDIBLE] or

88
00:05:12,100 --> 00:05:16,710
that the two variables and achievements
are all the seven variables get attached.

89
00:05:16,710 --> 00:05:19,350
But of course, as we have seen
that is not the best thing to do.

90
00:05:19,350 --> 00:05:22,410
Here we are using it because
it's a one off command.

91
00:05:22,410 --> 00:05:26,480
So, once you do that, attach achievement,
agent IQ variable available to you and

92
00:05:26,480 --> 00:05:27,170
then if you say,

93
00:05:27,170 --> 00:05:32,040
plot h versus IQ, then you just get the
part that you see on the left hand side.

94
00:05:33,530 --> 00:05:39,960
Now, if you give the same command plot
HIQ as an argumentative system of time,

95
00:05:39,960 --> 00:05:45,920
then you'll be told how much time is
spent in different ways in the user time,

96
00:05:45,920 --> 00:05:47,900
system time, and the total elapsed time.

97
00:05:48,980 --> 00:05:53,330
And what we also see here is how to save
the file, which we haven't seen before.

98
00:05:53,330 --> 00:05:58,750
So you can give the PNG commands to save
the file the plot has a png command.

99
00:05:58,750 --> 00:06:03,830
Okay, so we see how a simple
command system.time can be given.

100
00:06:03,830 --> 00:06:08,280
But now, within the plot command
the the system would have done

101
00:06:08,280 --> 00:06:09,660
several different kind of things.

102
00:06:09,660 --> 00:06:15,160
So, if my just to see, what it is
that running, when that happens.

103
00:06:15,160 --> 00:06:18,220
So that is where the Rprof
command can come in.

104
00:06:18,220 --> 00:06:19,628
So, you can say that Rprof or

105
00:06:19,628 --> 00:06:23,330
plotprof, plotprof is the file
name that we are using here.

106
00:06:23,330 --> 00:06:28,340
And then once you start then give the
system.time command with the plot of age

107
00:06:28,340 --> 00:06:29,250
versus IQ as the command.

108
00:06:29,250 --> 00:06:33,950
As you see that the times that it has
returned are somewhat similar not too

109
00:06:33,950 --> 00:06:37,720
far different from the earlier
times that we have got.

110
00:06:37,720 --> 00:06:39,240
And then after that single command,

111
00:06:39,240 --> 00:06:44,300
we turn up the profiling by providing
the null argument to Rprof.

112
00:06:44,300 --> 00:06:47,360
And then,
now if you run some of the Rprof on that,

113
00:06:47,360 --> 00:06:49,850
you see that there
are several lines of output.

114
00:06:49,850 --> 00:06:53,970
So, it called the GC and the axis and

115
00:06:53,970 --> 00:06:58,390
it gives you the system time for
each of them and axis.default and so on.

116
00:06:58,390 --> 00:07:02,700
So, of course if this were a real program,
then it'll go into many,

117
00:07:02,700 --> 00:07:04,180
many different subroutines and

118
00:07:04,180 --> 00:07:07,760
functions, and for each of them you'll
get to know how much time we'll spend.

119
00:07:07,760 --> 00:07:11,870
And then you can figure out which of them
is taking much more time than others where

120
00:07:11,870 --> 00:07:14,010
you may want to decrease
that a little bit.

121
00:07:15,400 --> 00:07:16,570
On the right hand side bottom,

122
00:07:16,570 --> 00:07:19,380
you see just the output of
the plot profile itself.

123
00:07:19,380 --> 00:07:22,130
And you can see that it doesn't
have anything of which you

124
00:07:22,130 --> 00:07:27,170
can make much sense in terms of the time
that was was being spent in each of them.

125
00:07:27,170 --> 00:07:30,490
So this profiling is a very
useful thing in order to

126
00:07:30,490 --> 00:07:35,220
figure out where you can be improving
your program by benchmarking it.

127
00:07:36,280 --> 00:07:39,070
Another important thing that we saw
in best programming practices is

128
00:07:39,070 --> 00:07:40,650
that you need to do is debugging.

129
00:07:40,650 --> 00:07:44,980
And again with R debugging can
be done in many different ways.

130
00:07:44,980 --> 00:07:48,490
RStudio has very elaborate
debugging framework, and

131
00:07:48,490 --> 00:07:52,530
I encourage you to use RStudio and take
a look at the debugging framework there.

132
00:07:52,530 --> 00:07:58,080
But there are also some basic R
commands available in the base of,

133
00:07:58,080 --> 00:07:59,400
base R that you can use.

134
00:07:59,400 --> 00:08:01,070
So trace back.

135
00:08:01,070 --> 00:08:05,750
If run trace back, then it's going to tell
you not only where the error occurred, but

136
00:08:05,750 --> 00:08:09,550
also which subroutine was
running when that was called.

137
00:08:09,550 --> 00:08:12,589
And go on all the way back,
giving you a complete trace.

138
00:08:13,700 --> 00:08:17,470
Similarly, you can dump entire
frames when an error is

139
00:08:18,580 --> 00:08:22,600
encountered by using the options
error equal to dump frames command.

140
00:08:22,600 --> 00:08:26,650
And then the debugger can be called at
that low level to see what all can be

141
00:08:26,650 --> 00:08:31,820
done, inspect the entire frame and
do what you would like to do with it.

142
00:08:31,820 --> 00:08:37,030
Similarly, we saw earlier that many
people tend to ignore warnings.

143
00:08:37,030 --> 00:08:40,750
If there is no error and there are few
warnings, they would simply go ahead and

144
00:08:40,750 --> 00:08:44,070
say oh, these are warnings, so I don't
need to worry about that and proceed.

145
00:08:44,070 --> 00:08:45,730
Which is not a good thing.

146
00:08:45,730 --> 00:08:49,670
So there's a way to convert
the warnings into errors as well, and

147
00:08:49,670 --> 00:08:52,180
that you can do using options(warn=2),
there.

148
00:08:53,520 --> 00:08:58,630
And then similarly, debug and a file name
can be given to get information from

149
00:08:58,630 --> 00:09:01,650
whatever was happening in that particular
function when they hit a [INAUDIBLE].

150
00:09:01,650 --> 00:09:07,240
Now what we'll see here is
accessing some built-in data sets.

151
00:09:07,240 --> 00:09:11,960
Remember, when we tried to access
achievement there, we could not access it

152
00:09:11,960 --> 00:09:16,200
because it belonged to a particular
package called on by which was not loaded.

153
00:09:16,200 --> 00:09:18,280
And we couldn't get rid of
it unless we loaded that.

154
00:09:18,280 --> 00:09:22,580
But there are a set of data
sets that are already there in

155
00:09:22,580 --> 00:09:24,360
R which you can directly access.

156
00:09:24,360 --> 00:09:28,300
And one way to find out which these
are by just giving the data command with

157
00:09:28,300 --> 00:09:29,950
empty parenthesis.

158
00:09:29,950 --> 00:09:32,340
And then you'll get
a list of such data sets.

159
00:09:32,340 --> 00:09:37,910
So here we'll see for instance if
we load the AirPassengers data set.

160
00:09:37,910 --> 00:09:40,590
And then giving question mark,
AirPassengers gives us

161
00:09:40,590 --> 00:09:43,990
more details about that data set to,
what are the columns involved?

162
00:09:43,990 --> 00:09:45,630
What is the length of the data set?

163
00:09:45,630 --> 00:09:48,120
And some additional detail on that.

164
00:09:48,120 --> 00:09:50,400
And you can even edit such a data set.

165
00:09:50,400 --> 00:09:52,540
It'll make a copy for you.

166
00:09:52,540 --> 00:09:57,780
And in this particular dataset, for
instance there are 144 columns and, and

167
00:09:57,780 --> 00:10:04,260
using the usual you can massage it into
a 12 by 12 array and, work on that.

168
00:10:04,260 --> 00:10:06,800
And you can do a pair spot on that and
so on.

169
00:10:06,800 --> 00:10:12,130
Another dataset that is useful,
to look at because it's very small and,

170
00:10:12,130 --> 00:10:14,000
gives only two dimensions, is, cars.

171
00:10:14,000 --> 00:10:18,350
So you can simply say data,
cars, and the plot cars.

172
00:10:18,350 --> 00:10:21,430
It'll just go ahead and
plot one column versus the other.

173
00:10:22,430 --> 00:10:25,430
Similarly another data set is
you should see bad emissions,

174
00:10:25,430 --> 00:10:28,700
you should try doing that if you plot
using bad emissions you'll find that

175
00:10:28,700 --> 00:10:31,520
is a more complex plot that comes about.

176
00:10:31,520 --> 00:10:36,190
And we'll return to that particular
point soon, because plotting for

177
00:10:36,190 --> 00:10:40,370
each different argument can
invoke different methods,

178
00:10:40,370 --> 00:10:42,860
depending on what it is
that you're trying to plot.

179
00:10:42,860 --> 00:10:45,680
And there are indirect ways of telling R,

180
00:10:45,680 --> 00:10:49,520
what it is that you want to plot,
by just a single plot command.

181
00:10:49,520 --> 00:10:54,420
So, for plotting complex data set,
it is a very useful thing to do.

182
00:10:54,420 --> 00:10:57,260
Before that, let's look at some
of the plots that you could do.

183
00:10:57,260 --> 00:11:01,110
So you can simply do xy plot, or

184
00:11:01,110 --> 00:11:05,150
do a histogram or
a dot chart by just calling those names.

185
00:11:05,150 --> 00:11:10,520
There advantage with R is again, that
these are orderly commands, which means

186
00:11:10,520 --> 00:11:14,110
that if you give it the minimal arguments,
it will do the minimal things.

187
00:11:14,110 --> 00:11:17,870
But then it can also take lots and
lots of additional arguments which you

188
00:11:17,870 --> 00:11:21,480
can find out using help, what it is
that they're going to be able to do.

189
00:11:23,600 --> 00:11:28,688
So, now, you can see a lot of basic
statistics using simply attaching cars and

190
00:11:28,688 --> 00:11:33,410
mean of speed which is one of
the variable, or max of the speed or

191
00:11:33,410 --> 00:11:40,180
summary and doing various things like
dotchart of it or doing a barplot of it.

192
00:11:40,180 --> 00:11:43,250
And you can give various colors to it or
you can get pie(speed).

193
00:11:43,250 --> 00:11:45,850
So these are things that
you can explore on you own,

194
00:11:45,850 --> 00:11:48,550
I'm just showing some of them to you so
you can get it.

195
00:11:48,550 --> 00:11:52,720
And then it is time that you don't want
the variables from cars to be there,

196
00:11:52,720 --> 00:11:54,270
you can simply detach those.

197
00:11:55,700 --> 00:11:57,950
Here is one such data set which may seem.

198
00:11:59,250 --> 00:12:01,720
it, it is a four dimensional array, and

199
00:12:01,720 --> 00:12:05,340
when you simply say plot(Titanic),
this is the plot that you'll get.

200
00:12:05,340 --> 00:12:10,740
And it is about such a plot that we'll
see more details, how to attach specific

201
00:12:11,970 --> 00:12:16,800
configurations of given data set that
you have to simple plot commands.

202
00:12:16,800 --> 00:12:20,620
So, the default plot meth,
method is what you will define.

203
00:12:20,620 --> 00:12:23,540
And then you can associate such
methods with the data set.

204
00:12:23,540 --> 00:12:26,420
And more generically with
the objects to decide what

205
00:12:26,420 --> 00:12:28,280
the default behavior should be.

206
00:12:28,280 --> 00:12:31,970
And that are, those are some of
the things that we'll be getting into,

207
00:12:31,970 --> 00:12:36,210
into the next set of the R lectures.

208
00:12:36,210 --> 00:12:39,350
So, we'll be looking at something
called astRowRap, where we use

209
00:12:39,350 --> 00:12:44,450
specific data sets from astronomy, and
attach specific plot methods to it.

210
00:12:44,450 --> 00:12:49,210
And swirl which is another simple
way of learning R for beginners.

211
00:12:49,210 --> 00:12:50,792
So we'll see more of that next time.