1 00:00:08,577 --> 00:00:10,790 Welcome back. 2 00:00:10,790 --> 00:00:14,980 This module, we are going to talk about Basic Natural Language Processing, and 3 00:00:14,980 --> 00:00:20,870 how it relates to the Text Mining in Python that we have been talking about. 4 00:00:20,870 --> 00:00:22,800 So, what is Natural Language? 5 00:00:22,800 --> 00:00:27,010 Well, any language that is used in everyday communication by humans 6 00:00:27,010 --> 00:00:28,610 is natural language. 7 00:00:28,610 --> 00:00:32,330 As compared to something that is artificial language or 8 00:00:32,330 --> 00:00:34,940 a computer language like Python. 9 00:00:34,940 --> 00:00:38,660 Languages such as English, or Chinese, or Hindi, or Russian, or 10 00:00:38,660 --> 00:00:40,810 Spanish are all natural languages. 11 00:00:40,810 --> 00:00:46,000 But you know, also the language we use in short text messages or 12 00:00:46,000 --> 00:00:50,100 on tweets is also, by this definition natural language, isn't it? 13 00:00:51,250 --> 00:00:53,420 So then we have to kind of address these as well. 14 00:00:54,930 --> 00:00:57,130 So then, what is Natural Language Processing? 15 00:00:57,130 --> 00:01:00,680 Any computation or manipulation of natural language 16 00:01:00,680 --> 00:01:04,710 to get some insights about how words mean and 17 00:01:04,710 --> 00:01:08,280 how sentences are constructed is natural language processing. 18 00:01:10,050 --> 00:01:15,490 One thing to consider when we look at natural language is that these evolve. 19 00:01:15,490 --> 00:01:17,176 For example, new words get added. 20 00:01:17,176 --> 00:01:19,981 Like selfie or photobomb. 21 00:01:21,410 --> 00:01:23,020 Old words lose popularity. 22 00:01:24,100 --> 00:01:25,990 How often have you used thou shalt? 23 00:01:27,210 --> 00:01:28,470 Meanings of words change. 24 00:01:29,800 --> 00:01:34,670 Words such as learn in Old English meant exactly opposite of what it means now. 25 00:01:34,670 --> 00:01:35,460 It used to mean teach. 26 00:01:36,940 --> 00:01:39,936 And then language rules themselves may change. 27 00:01:39,936 --> 00:01:43,069 So for example, in Old English the position of the verb was at 28 00:01:43,069 --> 00:01:46,824 the end of the sentence, rather than the middle as we come to know today. 29 00:01:49,020 --> 00:01:52,010 So when we talk about NLP tasks, what do we mean? 30 00:01:53,700 --> 00:01:56,217 .... It could mean as simple as counting 31 00:01:56,217 --> 00:02:02,115 the words or counting the frequency of a word or finding unique words in a corpus, 32 00:02:02,115 --> 00:02:05,585 and then build on to find sentence boundaries or 33 00:02:05,585 --> 00:02:09,748 parts of speech to tag a sentence with its part of speech, 34 00:02:09,748 --> 00:02:15,822 parse the sentence structure, try to understand more grammatical constructs and 35 00:02:15,822 --> 00:02:19,590 see whether they apply for a particular sentence. 36 00:02:20,990 --> 00:02:24,490 Identify semantic roles of how these words play. 37 00:02:24,490 --> 00:02:30,450 For example, if you have a sentence like Mary loves John. 38 00:02:30,450 --> 00:02:33,370 Then you know that Mary is the subject. 39 00:02:33,370 --> 00:02:38,000 John is the object, and love is the verb that connects them. 40 00:02:39,510 --> 00:02:44,070 You have the other NLP tasks like identifying entities in a sentence. 41 00:02:44,070 --> 00:02:48,250 So this is called name entity recognition, and 42 00:02:48,250 --> 00:02:54,640 in our previous example of Mary loves John, Mary and John are the two entities. 43 00:02:54,640 --> 00:02:57,130 Both persons in that sentence. 44 00:02:58,260 --> 00:03:01,944 And then you could have more complicated, 45 00:03:01,944 --> 00:03:08,500 more complex tasks like finding which pronoun refers to which entity. 46 00:03:08,500 --> 00:03:11,710 This is called co-ref resolution, or co-reference resolution. 47 00:03:12,740 --> 00:03:18,100 And there are many, many more tasks that you would do for on free text. 48 00:03:19,220 --> 00:03:23,360 The challenge is how to do that in an efficient manner, and 49 00:03:23,360 --> 00:03:26,156 how it applies to overall text mining. 50 00:03:26,156 --> 00:03:29,380 And you're going to see some of those in the next few videos.