1
00:00:00,008 --> 00:00:05,510
[MUSIC]. 

2
00:00:05,510 --> 00:00:07,971
Welcome back. 
So I want to talk about in this segment 

3
00:00:07,971 --> 00:00:10,975
what this term data science actually 
means actually means. 

4
00:00:10,975 --> 00:00:14,303
So, you know, you'll see these quotes 
around the web so we in Fortune Magazine 

5
00:00:14,303 --> 00:00:17,631
about Data Science being the hot new gig 
and Tech and Hal Varian who was chief, 

6
00:00:17,631 --> 00:00:21,219
Google's chief economist in the New York 
Times in 2009, which was a while ago now 

7
00:00:21,219 --> 00:00:24,443
talked about statistics being the next 
sexy job and described it as the you 

8
00:00:24,443 --> 00:00:27,563
know, the ability to take data to 
understand it, process it, and extract 

9
00:00:27,563 --> 00:00:35,910
value from it and communicate it. 
That's going to be hugely important. 

10
00:00:35,910 --> 00:00:39,312
Another person who's prolific In thinking 
and writing in this space is Mike 

11
00:00:39,312 --> 00:00:42,606
Driscoll whose the CEO of a company 
called Metamarkets and he talks about 

12
00:00:42,606 --> 00:00:45,954
there the data science there as it's 
practiced is as a cloacal view of it is a 

13
00:00:45,954 --> 00:00:52,110
blend of Red-Bull-fueled hacking and 
espresso-inspired statistics. 

14
00:00:52,110 --> 00:00:55,165
And maybe another quote of his is the 
Data sciences is the civil engineering of 

15
00:00:55,165 --> 00:00:58,760
data. 
Who, you know, whose accolades persist as 

16
00:00:58,760 --> 00:01:01,940
practical knowledge, as well as a 
theoretical understanding. 

17
00:01:01,940 --> 00:01:06,632
And so there's balance between pragmatism 
and theory is something we'll come back 

18
00:01:06,632 --> 00:01:09,374
to. 
So another perspective on this that you 

19
00:01:09,374 --> 00:01:12,302
should be familiar with is this Venn 
diagram that made the rounds several 

20
00:01:12,302 --> 00:01:16,589
years ago by Drew Conway. 
And what he, his point was that Data 

21
00:01:16,589 --> 00:01:21,530
Science is probably the mix of three 
different sort of areas. 

22
00:01:21,530 --> 00:01:24,390
One is hacking skills you know, 
programming expertise. 

23
00:01:24,390 --> 00:01:29,220
Another is the academic view you know, 
the math and statistics knowledge. 

24
00:01:29,220 --> 00:01:32,720
And then the third that he added is this 
notion of substantive expertise. 

25
00:01:32,720 --> 00:01:36,090
And what he meant by this is kind of a 
deep investment with the data. 

26
00:01:36,090 --> 00:01:39,130
So if you think about Your typical IT 
shop. 

27
00:01:39,130 --> 00:01:42,681
They're typically building tools, for 
other people to use, to actually analyze 

28
00:01:42,681 --> 00:01:47,570
the data, but they don't necessarily do 
the analysis themselves. 

29
00:01:47,570 --> 00:01:51,218
And, in, you know, a data scientist, in 
contrast, may be a participant in tool 

30
00:01:51,218 --> 00:01:54,923
building, but they're going to also, sort 
of, dive deeply into the data and do the 

31
00:01:54,923 --> 00:01:59,652
analysis themselves. 
And then, that's one of the ways that I 

32
00:01:59,652 --> 00:02:04,518
like to interpret Drew's blue bullet here 
of, of substituted expertise. 

33
00:02:04,518 --> 00:02:08,109
Okay, and then he also sort of fills in 
the gaps between these, talking about 

34
00:02:08,109 --> 00:02:12,099
that I'm not sure that I'd agree that 
this is necessarily traditional research, 

35
00:02:12,099 --> 00:02:15,747
but perhaps it is, is applying the 
statistical knowledge to a particular 

36
00:02:15,747 --> 00:02:19,566
domain is where research Comes in and at 
some deep theory plus some pragmatic 

37
00:02:19,566 --> 00:02:25,580
programming makes you a machine learning 
expert. 

38
00:02:25,580 --> 00:02:27,590
And then he sort of refers to this as the 
danger zone. 

39
00:02:27,590 --> 00:02:31,620
Where you sort of know enough about the 
domain to be dangerous. 

40
00:02:31,620 --> 00:02:34,028
And you know enough hacking to be 
dangerous but you don't know how to 

41
00:02:34,028 --> 00:02:38,580
ground your analysis in. 
Proper theorem. 

42
00:02:38,580 --> 00:02:41,880
And you know, some of my colleagues like 
to joke, the computer scientists don't 

43
00:02:41,880 --> 00:02:45,284
understand error bars, right. 
And that's, that's maybe what they're 

44
00:02:45,284 --> 00:02:48,050
referring to here, okay. 
So fine. 

45
00:02:49,090 --> 00:02:52,254
So what do data scientists actually do? 
Well, some more quotes here from, you 

46
00:02:52,254 --> 00:02:55,602
know, from EMC who is a company who 
acquired a company called Green Plum and 

47
00:02:55,602 --> 00:02:58,950
have a Reason we have a pretty 
significant initiative in data science, 

48
00:02:58,950 --> 00:03:04,970
both related to chronicle green plumb as 
well as other products. 

49
00:03:04,970 --> 00:03:08,740
So we need to find the nuggets in data 
and then explain it to business leaders. 

50
00:03:08,740 --> 00:03:11,090
And I like this, you know, and then 
explain it to business leaders. 

51
00:03:11,090 --> 00:03:13,352
Something else we'll come back to that I 
mentioned previously is that 

52
00:03:13,352 --> 00:03:16,410
communicating results is a critical part 
of this. 

53
00:03:16,410 --> 00:03:18,622
process, it's not just getting the 
result. 

54
00:03:18,622 --> 00:03:21,846
Okay, and then another, view of this, 
which is sort of interesting, is that 

55
00:03:21,846 --> 00:03:26,580
data scie-, DJ Patil, talked about, data 
scientist tend to be hard scientist. 

56
00:03:26,580 --> 00:03:29,450
Maybe coming from a physics background, 
who have a strong mathematical background 

57
00:03:29,450 --> 00:03:32,156
and computing skills, and, you know, come 
from a discipline, in which survival 

58
00:03:32,156 --> 00:03:34,780
depends on getting the most from the 
data, they're, really, used to, sort of, 

59
00:03:34,780 --> 00:03:39,298
torturing the data, to extract every last 
ounce of value out of it. 

60
00:03:39,298 --> 00:03:42,623
Out of it. 
This, you know, DJ has an applied math 

61
00:03:42,623 --> 00:03:47,850
background so he's maybe coming from that 
perspective. 

62
00:03:48,950 --> 00:03:51,817
So Mike Driscoll also talks about the 
three skills of data geeks which are 

63
00:03:51,817 --> 00:03:54,966
statistics, this thing data munging and 
we'll kind of, you know, this funny word 

64
00:03:54,966 --> 00:03:59,330
munging, we'll kind of return to this. 
You'll see a lot of this sort of 

65
00:03:59,330 --> 00:04:02,450
colloquial Language and I'll give my 
perspective on what I think that tells us 

66
00:04:02,450 --> 00:04:06,700
about the, the state of the world in Data 
Science. 

67
00:04:06,700 --> 00:04:09,836
And but, what he means by this, as you 
can imagine, is sort of parsing data, and 

68
00:04:09,836 --> 00:04:12,727
scraping data from the web, and 
converting to, between different file 

69
00:04:12,727 --> 00:04:16,255
formats effeciently, and not getting hung 
up on these the kind of friction that you 

70
00:04:16,255 --> 00:04:22,705
deal with when you are working with large 
and, and, and heterogeneous data sets. 

71
00:04:22,705 --> 00:04:24,650
Alright? 
So a data scientist is someone who is 

72
00:04:24,650 --> 00:04:27,940
very comfortable in that environment and 
is able to sort of work nimbly even when 

73
00:04:27,940 --> 00:04:30,870
things aren't very clean. 
Okay. 

74
00:04:30,870 --> 00:04:34,220
And then finally, visualization is 
another sexy skill, right? 

75
00:04:34,220 --> 00:04:38,919
The ability to communicate the results 
through visualization, alright. 

76
00:04:38,919 --> 00:04:42,135
So another quote now from Jeffrey Stanton 
who teaches a course in data science at 

77
00:04:42,135 --> 00:04:46,638
Syracuse and has, was involved in one of 
the earlier programs in data science. 

78
00:04:46,638 --> 00:04:49,190
Talks about the emerging area of work 
concerned with the collection, 

79
00:04:49,190 --> 00:04:52,138
preparation, analysis, visualization, 
management and preservation of large 

80
00:04:52,138 --> 00:04:55,927
collections of information. 
And I think one thing interesting about 

81
00:04:55,927 --> 00:04:58,850
His perspective includes the word 
preservation. 

82
00:04:58,850 --> 00:05:03,335
While the preparation the analytics and 
visualization are the three tenants that 

83
00:05:03,335 --> 00:05:08,740
you see quite often is something that we 
like in this course as well. 

84
00:05:08,740 --> 00:05:11,446
Jeffrey goes one step further and talks 
about preservations even after you're 

85
00:05:11,446 --> 00:05:15,445
done communicating the results. 
What do you do with the data long term? 

86
00:05:15,445 --> 00:05:18,145
Part of the reason is he's got a 
background in library science and 

87
00:05:18,145 --> 00:05:21,645
information studies where they're very 
concerned with the curiation of data and 

88
00:05:21,645 --> 00:05:24,995
so that's something we're probably not 
going to emphasize in this course but its 

89
00:05:24,995 --> 00:05:30,135
definately part of the overall data life 
cycle if you will. 

90
00:05:30,135 --> 00:05:35,237
Okay. 
So another quote from thinker in this 

91
00:05:35,237 --> 00:05:38,010
space is Hillary Mason, the chief 
scientist at bit.ly, and so she says a 

92
00:05:38,010 --> 00:05:41,300
data scientist is someone who can obtain, 
scrub explore, model and interpret data, 

93
00:05:41,300 --> 00:05:45,909
you know, blending hacking, statistics 
and machine learning. 

94
00:05:45,909 --> 00:05:47,923
So we saw that before in Drew Conway's 
diagram. 

95
00:05:47,923 --> 00:05:51,068
There's blending. 
and data scientists are not only adept at 

96
00:05:51,068 --> 00:05:54,840
working with data, but can appreciate 
data itself as a first-class product. 

97
00:05:54,840 --> 00:05:58,232
And I think she's talking about there 
maybe is being able to organize the data 

98
00:05:58,232 --> 00:06:02,415
and actually produce something that's 
usable by other people. 

99
00:06:02,415 --> 00:06:04,710
Right? 
Okay. 

100
00:06:04,710 --> 00:06:08,554
So having a quality data Resource a data 
asset that others can use to answer 

101
00:06:08,554 --> 00:06:12,770
questions is one of the outputs of data 
science and then you know she talks about 

102
00:06:12,770 --> 00:06:16,862
let's see scrubbing and we saw munging on 
the previous slide you'll see data jiu 

103
00:06:16,862 --> 00:06:22,850
jitsu people will say you know this, 
this. 

104
00:06:22,850 --> 00:06:25,990
The data scientist that I want to hire is 
really good at data jujitsu. 

105
00:06:25,990 --> 00:06:29,122
And we don't, I, I, I, on, on, in a 
couple of segments I'll talk about, or 

106
00:06:29,122 --> 00:06:34,440
the next segment I think I'll talk about, 
what I think this means. 

107
00:06:34,440 --> 00:06:38,334
Okay, so to summarize what we talked 
about, there's perhaps three overarching 

108
00:06:38,334 --> 00:06:42,917
tasks involved in data science. 
And those are, you know, preparing to run 

109
00:06:42,917 --> 00:06:46,397
some sort of statistical analysis, 
actually running that statistical 

110
00:06:46,397 --> 00:06:51,080
analysis, and then interpreting and 
communicating the results. 

111
00:06:51,080 --> 00:06:54,801
And this phase here, this preparing to 
run a model, is where we see all that 

112
00:06:54,801 --> 00:06:59,620
data munging, cleaning, manipulating, 
integrating, and so on. 

113
00:06:59,620 --> 00:07:06,160
All right. 
So another view that I like to take is 

114
00:07:06,160 --> 00:07:10,384
that data science is really about data 
products, Producing data products that 

115
00:07:10,384 --> 00:07:14,585
you may use yourself or that others may 
use. 

116
00:07:14,585 --> 00:07:17,660
Okay, and so, what do I mean by data 
products? 

117
00:07:17,660 --> 00:07:20,170
Well this could be data driven 
applications. 

118
00:07:20,170 --> 00:07:23,005
So if you think about a spell checker, 
alright this is not just a piece of code 

119
00:07:23,005 --> 00:07:26,420
This is not just a piece of software that 
does something. 

120
00:07:26,420 --> 00:07:31,579
This is only enabled by a dictionary of 
words and a dictionary of misspelled word 

121
00:07:31,579 --> 00:07:35,455
actually. 
Okay and similarly a machine translation 

122
00:07:35,455 --> 00:07:39,195
where you can type a sentence in French 
and have it automatically translated into 

123
00:07:39,195 --> 00:07:42,979
Arabic. 
Relies on not just clever algorithms. 

124
00:07:44,310 --> 00:07:50,060
But rather an enormous corpus of French 
texts and Arabic texts, okay. 

125
00:07:50,060 --> 00:07:52,830
A second kind of data product is 
interactive visualizations. 

126
00:07:52,830 --> 00:07:55,495
we saw this example with the Google flu 
application, and there's a suite of 

127
00:07:55,495 --> 00:07:58,160
visualizations on the web that I'm not 
going to show right now called that are 

128
00:07:58,160 --> 00:08:01,390
associate with the global burden of 
disease. 

129
00:08:01,390 --> 00:08:04,700
And these are visualizations produced by 
the institute for. 

130
00:08:04,700 --> 00:08:07,410
Health measures here at the University of 
Washington. 

131
00:08:07,410 --> 00:08:09,593
And what I like about it is, it's an 
example of where they've done a lot of 

132
00:08:09,593 --> 00:08:11,636
research. 
But then the output was not just a 

133
00:08:11,636 --> 00:08:14,510
research paper, although they wrote 
plenty of those as well. 

134
00:08:14,510 --> 00:08:16,989
It was these interactive visualizations 
that allowed you to explore the data as 

135
00:08:16,989 --> 00:08:19,425
well. 
And so this, I think, captures this 

136
00:08:19,425 --> 00:08:23,325
notion of producing, not just the answer 
Or not just a paper in, in perhaps this 

137
00:08:23,325 --> 00:08:27,805
case, but a data product that is usable 
by others. 

138
00:08:27,805 --> 00:08:31,046
Okay. 
And then finally, another kind of data 

139
00:08:31,046 --> 00:08:34,220
product might be an online data, database 
of some kind that others can actually use 

140
00:08:34,220 --> 00:08:37,420
to can query and answer their own 
questions. 

141
00:08:37,420 --> 00:08:39,695
And so maybe there's not necessarily a 
visualization component, but the work 

142
00:08:39,695 --> 00:08:42,850
that goes into producing these things, I 
would argue is part of data science. 

143
00:08:42,850 --> 00:08:45,880
And so this. 
Captures some of the Enterprise data 

144
00:08:45,880 --> 00:08:50,520
warehouse work and software and effort, 
of which there's a lot. 

145
00:08:50,520 --> 00:08:53,267
And including business intelligence work 
and I'll, and in a couple of segments, 

146
00:08:53,267 --> 00:08:58,242
I'll try to differentiate those two. 
Another example of an online database 

147
00:08:58,242 --> 00:09:01,608
from science as opposed to business is 
the Sloan Digital Sky Survey, which I'll 

148
00:09:01,608 --> 00:09:05,290
talk about in more detail in a couple of 
segments. 

149
00:09:05,290 --> 00:09:06,790
So, again just to summarize here whats in 
red. 

150
00:09:06,790 --> 00:09:09,140
Data science is not just about building 
data products. 

151
00:09:09,140 --> 00:09:12,800
Rather it's about building data products, 
not just answering the questions once. 

152
00:09:12,800 --> 00:09:16,892
And what data products are our assets, 
visual assets that empower others to use 

153
00:09:16,892 --> 00:09:22,180
the data in new ways, and so they may 
help communicate results. 

154
00:09:22,180 --> 00:09:25,630
For example, Nate Silver built these maps 
or they may empower others to do their 

155
00:09:25,630 --> 00:09:30,098
own kind of analysis, say with a data 
warehouse, or with a visualization. 

156
00:09:30,098 --> 00:09:30,714
Alright.