The Naive Bayes algorithm is one of the most
important algorithms for text
classification. The intuition of the Naive
Bayes algorithm is really quite simple.
It's based on Bayes Rule, which we'll see
in a second, and it relies on a very
simple representation of the document
called the bag of words representation.
Let's see the intuition of the bag of
words representation. Imagine I have some
document that says, I love this movie,
it's sweet but with satirical humor. And
so, and our job. Is to, is to build this
function gamma, which takes the document
and returns a class. The class could be
positive. Or the class could be negative
in case of, of sentiment analysis. Which is it
a positive or negative? In order to solve
this task, one thing we might do is look
at individual words in the document, like
love or satirical or great. We might look
at all of the words. In some kinds of text
classification we're gonna look at all the
word, we're gonna look at every single
word. In other cases we'll look at just
some subset. If we were to look at a subset,
we might imagine that the document looks
something like this. It just looks like it
has the word love, and the word satirical,
and the word great and all the other words
have disappeared. Whether we use a subset
of words or all of the words in the
document. The bag of words
representation loses all the information
about the order of the words in the
document and all we represent about the
document is the set of words that occurred
and their accounts. So for example for the
previous document, we might represent the
document as just a vector of words: great,
love, recommend, laugh, happy. And for
each one account: great occurred twice,
love occurred twice, recommend occurred
once. And again, we can keep all of the
words in the document, and we'll often do
that or we can just keep some of the words
in the document if we have an idea that
some of the words are particularly
indicative cues. So the idea of the bag of
words' representation is that we're gonna
represent our document just by a list of
words. And there counts, and throw away
everything else about the document. Which
order the words occurred in, what font
they were in, anything else, and our
function will, our function gamma, our
classifier, will take that representation,
and assign us a class positive or
negative. And this applies, I've shown it
to you for the two class problem of
sentiment analysis, positive or negative
sentiment. But this applies for all sorts
of document classification tasks. So I
might have some document I need to
classify into a different computer science
topic because I'm building an online
library of computer science papers. Or I'm
giving advice on computer science topics.
So I have some text, some, some document
here with words like parser or language or
label or translation and I wanna know
which aspect of computer science it should
go in so I can file my paper automatically
and a good text classifier should
automatically figure out that that's a,
that's a natural language processing
paper. So that's the intuition of the
naive base classifier.