1
00:00:00,780 --> 00:00:02,430
My name is Ashish Mahabal, and

2
00:00:02,430 --> 00:00:05,600
I'm going to be talking about
the R programming language today.

3
00:00:05,600 --> 00:00:09,350
Later on you will be using
R in many other models.

4
00:00:09,350 --> 00:00:13,090
Here what we'll be doing is looking
at only a few basic things in

5
00:00:13,090 --> 00:00:16,480
a slightly different, but there are many
tutorials available on R as well,

6
00:00:16,480 --> 00:00:19,420
but we'll be looking at how to install R,
how to.

7
00:00:19,420 --> 00:00:24,620
Do some very basic things with R, so
that you can hit the ground running.

8
00:00:24,620 --> 00:00:25,250
To do that,

9
00:00:25,250 --> 00:00:29,850
we'll start with some of the slightly
different things about R right away.

10
00:00:29,850 --> 00:00:32,930
And then come back to the basic
history of it and so on.

11
00:00:32,930 --> 00:00:36,340
So, here what you see is a simple
linear regression example.

12
00:00:36,340 --> 00:00:39,000
And R is of course wonderful for
doing lots of statistics.

13
00:00:39,000 --> 00:00:41,552
And this is just one of
the simpler examples.

14
00:00:41,552 --> 00:00:42,900
By the way,

15
00:00:42,900 --> 00:00:47,530
you start R by just saying R on
the command line prompt once you have R.

16
00:00:47,530 --> 00:00:53,700
Then what you see in the next line is
that we have assigned a vector which has

17
00:00:53,700 --> 00:00:59,220
three numbers, one, two and four to the
name of the vector here we are using is y.

18
00:00:59,220 --> 00:01:02,300
And in the next line we
are using the numbers one, two,

19
00:01:02,300 --> 00:01:05,500
three combining them and
assigning them to a vector x.

20
00:01:05,500 --> 00:01:07,050
So that is how the syntax is.

21
00:01:07,050 --> 00:01:10,420
For assignment we use the less than,

22
00:01:10,420 --> 00:01:14,380
followed by a dash as
the assignment character set.

23
00:01:14,380 --> 00:01:15,820
And c is used for

24
00:01:15,820 --> 00:01:19,840
combining, and you can combine a large
number of different things with that.

25
00:01:19,840 --> 00:01:21,770
But that is the main thing
you have to remember.

26
00:01:21,770 --> 00:01:24,410
So it's not the equal to sign
that you use for assignment.

27
00:01:24,410 --> 00:01:27,350
Equal to can also be used,
we can see that later, but

28
00:01:27,350 --> 00:01:31,860
what is really used is less than and dash,
so you should better get used to that.

29
00:01:32,860 --> 00:01:33,850
Then, in the next line,

30
00:01:33,850 --> 00:01:38,250
we see how the linear regression
example uses the linear model.

31
00:01:38,250 --> 00:01:41,600
And it takes argument y, tilde x.

32
00:01:41,600 --> 00:01:45,020
And again this is something you'll
see in a non standard thing.

33
00:01:45,020 --> 00:01:48,700
So what we are saying here is that
y is a dependent variable, and

34
00:01:48,700 --> 00:01:50,580
x is the variable on which it depends.

35
00:01:50,580 --> 00:01:53,030
And that is what one means by Y wiggle x.

36
00:01:53,030 --> 00:01:55,575
And then when you run
the linear model on that,

37
00:01:55,575 --> 00:02:00,550
then the output of that is being
captured in a variable called foo.

38
00:02:00,550 --> 00:02:01,610
It's not really a variable,

39
00:02:01,610 --> 00:02:04,760
it's an entire structure,
though it seems just like a variable.

40
00:02:04,760 --> 00:02:07,790
And if you then say simply
foo on the command line

41
00:02:07,790 --> 00:02:11,870
prompt you'll see what that
particular structure contains.

42
00:02:11,870 --> 00:02:14,710
So, initially it shows
you how the call is.

43
00:02:14,710 --> 00:02:19,940
It says that lm can be called
with a formula like y wiggle x.

44
00:02:19,940 --> 00:02:23,260
And then it goes ahead and tells you
what other things there are within

45
00:02:23,260 --> 00:02:25,570
the structure, like the coefficients here.

46
00:02:25,570 --> 00:02:31,180
The slope is 1.5 and
the intercept is minus 0.67.

47
00:02:31,180 --> 00:02:35,480
So this is how you will run
a simple statistical model.

48
00:02:35,480 --> 00:02:38,880
This is how you'll assign two
variables and two vectors and so on.

49
00:02:38,880 --> 00:02:42,810
And we'll of course be covering
more details of that later on.

50
00:02:44,330 --> 00:02:45,730
Here is another simple example.

51
00:02:45,730 --> 00:02:48,040
In this case it is of a plot.

52
00:02:48,040 --> 00:02:53,370
What we're doing here is using
the rnorm function, saying that we

53
00:02:53,370 --> 00:02:58,670
take 100 numbers in that which have a mean
of five and a standard deviation of one,

54
00:02:58,670 --> 00:03:03,220
and then assign a vector called
a that telling those 100 numbers.

55
00:03:03,220 --> 00:03:07,370
In the next line, we similarly assign
to b 200 numbers with the same mean and

56
00:03:07,370 --> 00:03:08,158
the same standard deviation.

57
00:03:08,158 --> 00:03:11,928
So again, the rnorm function
can be used just like that.

58
00:03:11,928 --> 00:03:16,030
It's all built in and
you can use it in a trivial fashion.

59
00:03:16,030 --> 00:03:19,910
But equally trivial is to get
say a histogram of the b vector.

60
00:03:19,910 --> 00:03:23,760
Simply saying hist b is going to
give you the plot that you see on

61
00:03:23,760 --> 00:03:24,850
the right hand side.

62
00:03:25,870 --> 00:03:29,690
And then combine can be used
with these vectors as well.

63
00:03:29,690 --> 00:03:35,990
So the next line you see that you are
combining 100 times the first vector and

64
00:03:35,990 --> 00:03:40,126
100 times the second vector and
assigning that to a variable called c.

65
00:03:40,126 --> 00:03:43,400
You see here is some kind of overloading
c's that combine function but

66
00:03:43,400 --> 00:03:46,280
what we are also using
the vector name called c so

67
00:03:46,280 --> 00:03:49,530
there's a lot of overloading,
also with how much you should do.

68
00:03:49,530 --> 00:03:53,870
You should decide based on whether
you really need to use those kinds of

69
00:03:53,870 --> 00:03:54,710
variable names.

70
00:03:55,810 --> 00:03:59,330
And then, in the next line you can
see that you can simply say length of

71
00:03:59,330 --> 00:04:03,940
c to find out that c is now
a vector of length 300.

72
00:04:03,940 --> 00:04:10,350
And if you simply type c, you'll see what
the variable c, or the vector c contains.

73
00:04:10,350 --> 00:04:15,180
And of course if you say help(c), it's
going to give you help not on the vector,

74
00:04:15,180 --> 00:04:18,270
but on the combined command,
so you try that out.

75
00:04:18,270 --> 00:04:18,820
And yes by the way,

76
00:04:18,820 --> 00:04:22,150
so, all these things, various
things that we'll be talking about,

77
00:04:22,150 --> 00:04:26,060
there'll be examples related to that,
there'll be exercises related to that.

78
00:04:26,060 --> 00:04:29,900
And you should definitely try many
of those out because doing is what

79
00:04:29,900 --> 00:04:32,230
will allow you to see how R functions.

80
00:04:32,230 --> 00:04:35,510
How the vagaries of R are and
how the strengths of R are.

81
00:04:37,220 --> 00:04:41,450
So, let's look at a little bit of
statistics how it's used in astronomics.

82
00:04:41,450 --> 00:04:45,880
So there are as many as 15,000
astronomical studies per year,

83
00:04:45,880 --> 00:04:49,800
and as many of 5% have the word
statistics in the abstract.

84
00:04:49,800 --> 00:04:52,990
Of course many,
many more have the word abstract,

85
00:04:52,990 --> 00:04:56,690
the word statistics in the rest of
the study and they use statistics a lot.

86
00:04:56,690 --> 00:05:00,820
Believe in an abstract,
as many as 5% have that.

87
00:05:00,820 --> 00:05:03,490
And 20% treat variable objects or
multivariate datasets.

88
00:05:03,490 --> 00:05:05,550
So clearly statistics is more and

89
00:05:05,550 --> 00:05:09,410
more increasingly used in astronomy
as well as in other sciences.

90
00:05:11,080 --> 00:05:15,660
But, if you look at what statistical
methods actually get used.

91
00:05:15,660 --> 00:05:19,780
You realize that a majority
of them are pre World War II.

92
00:05:19,780 --> 00:05:23,430
In fact, Fourier transform,
that is used very frequently in many,

93
00:05:23,430 --> 00:05:26,010
many places, is from the earliest,
early 19th century.

94
00:05:27,030 --> 00:05:29,330
Similarly, least squares
is from the same time, and

95
00:05:29,330 --> 00:05:31,580
chi square is from early 20th century.

96
00:05:31,580 --> 00:05:33,840
And the two other tests,
like Kolmogorov-Smirnov and

97
00:05:33,840 --> 00:05:37,600
Principal Component Analysis,
they're also from before World War II.

98
00:05:37,600 --> 00:05:40,830
Since then, a lot of developments
have happened in statistics.

99
00:05:40,830 --> 00:05:42,710
Many, many people do not use those but

100
00:05:42,710 --> 00:05:47,560
it is languages like R, that allow you
to use them in a very easy fashion.

101
00:05:47,560 --> 00:05:51,120
And hopefully you will get to learn
a lot of that in this module and

102
00:05:51,120 --> 00:05:53,410
a couple of other modules
that will be talking about R.

103
00:05:55,410 --> 00:05:58,670
Most major modules and
programming languages and

104
00:05:58,670 --> 00:06:04,810
packages have statistics Some examples
are Matlab, Mathematica, IDL and so on.

105
00:06:06,280 --> 00:06:08,340
So, what is special about R?

106
00:06:08,340 --> 00:06:12,170
Why would one try to go and
use R in particular?

107
00:06:12,170 --> 00:06:13,700
It's mainly because it's excellent for

108
00:06:13,700 --> 00:06:19,590
statistics of course, and a survey was
done in 2011 where 60 countries were,

109
00:06:19,590 --> 00:06:23,860
people from 60 countries were asked about
what is it that they use for data mining.

110
00:06:23,860 --> 00:06:25,590
And.
It turned out that 47% use R.

111
00:06:25,590 --> 00:06:28,990
So that's clearly a very big fraction.

112
00:06:28,990 --> 00:06:32,610
Given that there are many other
options that people have.

113
00:06:32,610 --> 00:06:38,870
And the one reason that is so, is because
R has been written by statisticians.

114
00:06:38,870 --> 00:06:42,820
It also has great layered graphics,
in terms of [INAUDIBLE] ggplot.

115
00:06:42,820 --> 00:06:46,120
We'll be seeing a little bit
of that later on, as well.

116
00:06:46,120 --> 00:06:50,560
A variety of GUIs are available too,
and interfaces, and of course,

117
00:06:50,560 --> 00:06:52,830
most important, is that it is free.

118
00:06:55,080 --> 00:06:58,980
So R actually comes from another
programming language called S.

119
00:06:58,980 --> 00:07:05,650
S was followed by S-plus in 1988, and
it was in 1993 that R came about.

120
00:07:05,650 --> 00:07:12,020
Ross Ihaka and Robert Gentleman wrote R,
the basic parts of it for laying S.

121
00:07:12,020 --> 00:07:15,550
And S was a commercial package,
and R was free.

122
00:07:15,550 --> 00:07:17,320
The current version is 3.1.1.

123
00:07:17,320 --> 00:07:22,820
That is what you should install
when you run the exercises here.

124
00:07:22,820 --> 00:07:26,420
And we'll see how object oriented R is,
and

125
00:07:26,420 --> 00:07:33,100
how you can use it from command line,
as well as from various scripts.

126
00:07:33,100 --> 00:07:38,980
So R just like S allows you to do lots and
lots of things various statistical tests.

127
00:07:38,980 --> 00:07:42,510
But also time series analysis that
is becoming more and more important.

128
00:07:42,510 --> 00:07:44,850
It can do classification and clustering.

129
00:07:44,850 --> 00:07:47,890
So when you install R from
the grand package, and

130
00:07:47,890 --> 00:07:52,830
we'll come to the grand website soon,
there are 15 recommended packages.

131
00:07:52,830 --> 00:07:55,510
And those cover a large
amount of statistics.

132
00:07:55,510 --> 00:07:59,360
And if you familiarize yourself just
with those then that'll cover most of

133
00:07:59,360 --> 00:08:00,500
your needs.

134
00:08:00,500 --> 00:08:06,680
So this a snapshot of how the various
recommended packages look.

135
00:08:06,680 --> 00:08:09,690
So, you have got smoothing and
classification and

136
00:08:09,690 --> 00:08:11,340
clustering as we mentioned.

137
00:08:11,340 --> 00:08:13,380
So, you should definitely
take a look at that.

138
00:08:14,930 --> 00:08:16,820
So, CRAN is what I mentioned earlier.

139
00:08:16,820 --> 00:08:19,660
It stands for
Comprehensive R Archive Network.

140
00:08:19,660 --> 00:08:22,850
And there are various mirror
sites available for that.

141
00:08:22,850 --> 00:08:27,250
But when you go there, you get to see that
beside those 15 recommended packages there

142
00:08:27,250 --> 00:08:32,340
are as many as 6000 different
contributed packages from people.

143
00:08:32,340 --> 00:08:34,680
Now, that really gives you
a much greater variety.

144
00:08:34,680 --> 00:08:38,470
The strength is that these are all
people contributed, and of course.

145
00:08:38,470 --> 00:08:40,960
One weakness is that there
is a bit of organic growth.

146
00:08:40,960 --> 00:08:46,056
Some of them get left out and then
the development does not continue, but

147
00:08:46,056 --> 00:08:49,850
the homogenous whole does allow you to

148
00:08:49,850 --> 00:08:54,680
use a large bit of idea
of statistical functions.

149
00:08:54,680 --> 00:08:58,680
A bioconductor.org's another
site where for bioinformatics it

150
00:08:58,680 --> 00:09:01,470
has a large number of packages available,
and that's another thing that you

151
00:09:01,470 --> 00:09:05,540
should definitely take a look at if
you are into that kind of thing.

152
00:09:07,070 --> 00:09:09,960
The other thing that I would like to
mention is that you should definitely get

153
00:09:09,960 --> 00:09:12,730
into using an interface and
editor, because that.

154
00:09:12,730 --> 00:09:14,610
Provides you a lot of boiler plate.

155
00:09:14,610 --> 00:09:18,390
Remember we talked earlier in
the best programming practices.

156
00:09:18,390 --> 00:09:20,510
How we should go to teach sheets and
boiler plates.

157
00:09:20,510 --> 00:09:22,804
So, that you should
[INAUDIBLE] here too for

158
00:09:22,804 --> 00:09:26,115
R you can find out what is the best
editor that you like to use.

159
00:09:26,115 --> 00:09:31,240
Rstudio.com which is mentioned last on
the slide is one of the best ones and

160
00:09:31,240 --> 00:09:32,689
I would definitely recommend that.

161
00:09:33,790 --> 00:09:37,250
More information related
to that is available

162
00:09:37,250 --> 00:09:41,710
on our sister material on the website,
so you should take a look at that.

163
00:09:43,460 --> 00:09:46,403
Downloading and installing R is
pretty straight forward just like

164
00:09:46,403 --> 00:09:52,600
various packages of R and here I've
just given you links for different

165
00:09:52,600 --> 00:09:57,058
flavors on Mac, how you would use it and
on Windows and other uniques a like.

166
00:09:57,058 --> 00:10:02,720
So again, it's going to that website,
downloading the package and a single click

167
00:10:02,720 --> 00:10:07,730
essentially, in most cases allow you
to install R in a very easy fashion.

168
00:10:09,950 --> 00:10:12,030
Running R is not difficult either.

169
00:10:12,030 --> 00:10:15,230
So what I would recommend initially
if you've not used R before is to

170
00:10:15,230 --> 00:10:18,540
make a directory, say R_work.

171
00:10:18,540 --> 00:10:20,040
Go to that directory.

172
00:10:20,040 --> 00:10:24,870
And then once you go there if you
hit R then you will be inside R and

173
00:10:24,870 --> 00:10:26,880
you will have an R prompt there.

174
00:10:28,230 --> 00:10:32,160
The most important thing is how to know,
to know how to start a package and

175
00:10:32,160 --> 00:10:35,530
the next most important thing is
of course how to get out of it.

176
00:10:35,530 --> 00:10:39,950
And the way you, you get out of R is
by pressing Q followed by parentheses.

177
00:10:39,950 --> 00:10:42,450
And then it'll give you a chance
to save your workspace.

178
00:10:42,450 --> 00:10:46,150
That can be quite important
if you want to reuse some of

179
00:10:46,150 --> 00:10:48,350
the commands that you have used.

180
00:10:48,350 --> 00:10:50,310
It can also save some of your data.

181
00:10:50,310 --> 00:10:51,930
So, during your session,

182
00:10:51,930 --> 00:10:56,460
if you have created some objects and
you want those objects to persist.

183
00:10:56,460 --> 00:11:00,510
You can also tell it that you would
like to use those objects later and

184
00:11:00,510 --> 00:11:02,500
those would go into R data.

185
00:11:02,500 --> 00:11:06,320
And there are ways to load your
earlier R data files later on.

186
00:11:08,700 --> 00:11:12,589
A few years ago we had also
created a GUI specifically for

187
00:11:12,589 --> 00:11:16,130
astronomists, it's still not, it's around.

188
00:11:16,130 --> 00:11:19,580
There you can provide input
terms of ASCII files or

189
00:11:19,580 --> 00:11:23,383
fits files or tables which are like XML.

190
00:11:23,383 --> 00:11:27,880
And then, you're allowed to pick various
columns from your input files and

191
00:11:27,880 --> 00:11:32,690
on those columns, then you can do
a variety of statistical functions.

192
00:11:32,690 --> 00:11:37,680
Here is a, current interface of that where
you see on the left side that you can

193
00:11:37,680 --> 00:11:40,690
choose either exploratory functions or
advanced functions or

194
00:11:40,690 --> 00:11:46,329
various export functions, and
how you can provide the input files.

195
00:11:48,570 --> 00:11:51,540
Here's a list of functions that
you can carry out in that.

196
00:11:51,540 --> 00:11:54,810
So there are some simple ones like
simply getting a box plot or histogram.

197
00:11:54,810 --> 00:11:58,430
But there are also advanced ones
like getting correlation matrix and

198
00:11:58,430 --> 00:11:59,870
covariance metrics.

199
00:11:59,870 --> 00:12:03,590
And then expert ones like H-clustering or
survival analysis.

200
00:12:03,590 --> 00:12:06,770
So other, another thing that I would like
to say is that you should definitely try

201
00:12:06,770 --> 00:12:10,120
out various examples that you'll
find in the follow up lectures.

202
00:12:10,120 --> 00:12:13,320
But you should also explore the net
because there's vast amount of

203
00:12:13,320 --> 00:12:16,370
material available through such GUIs.

204
00:12:16,370 --> 00:12:23,990
And that will help you a great deal.

205
00:12:23,990 --> 00:12:27,340
Finally, what you should be
aware of is how to get help, and

206
00:12:27,340 --> 00:12:30,440
help is available in
a variety of ways even in R.

207
00:12:30,440 --> 00:12:35,490
So, if you are looking for help on how to
solve something, simply saving help solve

208
00:12:35,490 --> 00:12:40,800
at the prompt is going to tell you how to
get help, what help is available on that.

209
00:12:40,800 --> 00:12:43,970
Similarly, doing something like
question search is going to

210
00:12:43,970 --> 00:12:46,570
provide you options on searching.

211
00:12:46,570 --> 00:12:50,270
And then if you want the help to
come not on the command line or

212
00:12:50,270 --> 00:12:54,380
within your window, but
in a separate browser then the way to

213
00:12:54,380 --> 00:12:58,861
start that functionality is to do
help.start followed by a parenthesis.

214
00:12:58,861 --> 00:13:03,310
And what that'll do is it will
open a browser window and

215
00:13:03,310 --> 00:13:05,920
bring up help material there.

216
00:13:05,920 --> 00:13:09,040
And then you can browse through it,
click and so on.

217
00:13:09,040 --> 00:13:09,890
Just like question,

218
00:13:09,890 --> 00:13:14,330
there is the additional help available
with question question, keyword.

219
00:13:14,330 --> 00:13:17,390
And when you do that, then you can find.

220
00:13:17,390 --> 00:13:19,070
Say, if you do question question matrix.

221
00:13:19,070 --> 00:13:21,370
Then you will get not just help on matrix.

222
00:13:21,370 --> 00:13:24,460
But also other commands which
may include the word, matrix.

223
00:13:24,460 --> 00:13:28,120
So you get to choose which
part of that you want.

224
00:13:28,120 --> 00:13:31,710
Because clearly terms like matrix
are going to be there in many,

225
00:13:31,710 --> 00:13:35,970
many different, functionalities,
different packages when R is involved.

226
00:13:37,470 --> 00:13:41,940
One other thing that you should
remember is that R is case sensitive.

227
00:13:41,940 --> 00:13:45,860
So, where capitals are needed,
you will want to use capitals.

228
00:13:45,860 --> 00:13:48,120
Here, I've shown an example of,

229
00:13:48,120 --> 00:13:53,360
getting environmental information on
what your home directory for R is.

230
00:13:53,360 --> 00:13:56,980
And the Sys keyword that you
are to use in such a case.

231
00:13:56,980 --> 00:13:58,810
The first S is capital there.

232
00:13:58,810 --> 00:14:00,835
So you would say, Sys.getenv.

233
00:14:00,835 --> 00:14:03,790
And in codes R_HOME.

234
00:14:03,790 --> 00:14:05,510
Whatever you have sited to and

235
00:14:05,510 --> 00:14:08,766
then you will be able to get
a what that information is.

236
00:14:08,766 --> 00:14:13,050
So R_HOME is the standard key word and
when if you have set it to

237
00:14:13,050 --> 00:14:17,560
something called R_HOME you will get that
back with the whole directory of course.

238
00:14:17,560 --> 00:14:20,690
And just like we saw that values
can be combined when you have

239
00:14:20,690 --> 00:14:21,620
assigned to a vector.

240
00:14:22,930 --> 00:14:25,460
Other bits like the environment variables,
or

241
00:14:25,460 --> 00:14:28,130
keywords, can also be similarly combined.

242
00:14:28,130 --> 00:14:33,000
The next example shows that if you want to
find out what your operating system is and

243
00:14:33,000 --> 00:14:37,230
your R_HOME is in one fell swoop,
then you can combine them with the c.

244
00:14:37,230 --> 00:14:40,650
And pass that on to the Sys.getenv.

245
00:14:40,650 --> 00:14:43,080
Now, in R everything in it is an object.

246
00:14:43,080 --> 00:14:46,420
And that is why combining
various things are very easy.

247
00:14:46,420 --> 00:14:49,730
So you can combine different objects,
pass them on as arguments and

248
00:14:49,730 --> 00:14:51,130
get other objects.

249
00:14:51,130 --> 00:14:53,580
We'll be, of course,
seeing more about that.

250
00:14:53,580 --> 00:14:56,940
Another very important function
is the summary function.

251
00:14:56,940 --> 00:14:58,580
Maybe you have created something or,

252
00:14:58,580 --> 00:15:02,170
if you have a variable vector
of various kinds of objects.

253
00:15:02,170 --> 00:15:07,540
Then you can provide you can get a summary
of what is contained in that by simply

254
00:15:07,540 --> 00:15:11,210
saying summary and the name of whatever
you're trying to get a summary of.

255
00:15:12,370 --> 00:15:16,670
So, again, there is a lot of help
available, and you should definitely be

256
00:15:16,670 --> 00:15:21,050
taking a look at various help routines,
making your life that much easier.

257
00:15:22,420 --> 00:15:26,470
Next time, we'll be covering in greater
detail the different ways in which you

258
00:15:26,470 --> 00:15:28,330
can do assignment and

259
00:15:28,330 --> 00:15:33,030
various things about, global variables,
et cetera, and about objects.

260
00:15:33,030 --> 00:15:37,670
How you create them and what additional
things you can do with objects.

261
00:15:37,670 --> 00:15:40,710
And one of the important concepts
in R is the dataframe, so

262
00:15:40,710 --> 00:15:42,460
we will be looking more at that.

263
00:15:43,590 --> 00:15:44,890
Will see that next time.