What is Bag of Words?

In our previous article of Natural Language Processing series, we discussed how to acquire a data, how to clean it and its preprocessing steps which are the required steps to perform this task. As we know our model takes only a numeric data as input, so we need to extract the numeric features from the cleaned and preprocessed textual data. Bag of words is a technique to extract the numeric features from the textual data.

How it Works?

Step 1: Data

Let's take 3 sentences:- "He is a good boy." - "She is a good girl."

  • "Girl and boy are good."
Step 2: Preprocessing

Here in this step we perform:- Lowercase the sentence - Remove stopwords

  • Perform tokenization
  • Perform stemming or lemmatization.
  • Words that we extractfrom each sentences:
    • From first sentence "good, boy"
    • From second sentence "good, girl"
    • From third sentence "girl, boy, good"
Step 3: Perform Bag of Words operation
Binary Bag of Words

Here in this step we assign value 1 for word that is present and 0 for word that is not present in the sentence.

  • In sentence 1:- good = 1 - boy = 1
    • girl = 0
  • In sentence 2:
    • good = 1
    • girl = 1
    • boy = 0
  • In sentence 3:
    • good = 1
    • boy = 1
    • girl = 1 ##### Bag of Words Here in this step we assign a value to each word according to their repetition in the sentence.
  • If a word present for two times in same sentences then we will assign the 2 for that word, otherwise we will do same as binary bag of words.

Code Implementation of Bag of Words 

import nltk
import re
from nltk.corpus import stopwords
from nltk.stem.porter import PorterStemmer
from nltk.stem import WordNetLemmatizer

paragraph =  """I have three visions for India. In 3000 years of our history, people from all over 
               the world have come and invaded us, captured our lands, conquered our minds. 
               From Alexander onwards, the Greeks, the Turks, the Moguls, the Portuguese, the British,
               the French, the Dutch, all of them came and looted us, took over what was ours. 
               Yet we have not done this to any other nation. We have not conquered anyone. 
               We have not grabbed their land, their culture, 
               their history and tried to enforce our way of life on them. 
               Why? Because we respect the freedom of others.That is why my 
               first vision is that of freedom. I believe that India got its first vision of 
               this in 1857, when we started the War of Independence. It is this freedom that
               we must protect and nurture and build on. If we are not free, no one will respect us.
               My second vision for India's development. For fifty years we have been a developing nation.
               It is time we see ourselves as a developed nation. We are among the top 5 nations of the world
               in terms of GDP. We have a 10 percent growth rate in most areas. Our poverty levels are falling.
               Our achievements are being globally recognised today. Yet we lack the self-confidence to
               see ourselves as a developed nation, self-reliant and self-assured. Isn't this incorrect?
               I have a third vision. India must stand up to the world. Because I believe that unless India 
               stands up to the world, no one will respect us. Only strength respects strength. We must be 
               strong not only as a military power but also as an economic power. Both must go hand-in-hand. 
               My good fortune was to have worked with three great minds. Dr. Vikram Sarabhai of the Dept. of 
               space, Professor Satish Dhawan, who succeeded him and Dr. Brahm Prakash, father of nuclear material.
               I was lucky to have worked with all three of them closely and consider this the great opportunity of my life. 
               I see four milestones in my career"""

Preprocessing Text Data

Link to Text Cleaning

Link to Preprocessing Textual Data

ps = PorterStemmer()
wordnet=WordNetLemmatizer()
sentences = nltk.sent_tokenize(paragraph)
corpus = []
for i in range(len(sentences)):
    review = re.sub('[^a-zA-Z]', ' ', sentences[i])
    review = review.lower()
    review = review.split()
    review = [ps.stem(word) for word in review if not word in set(stopwords.words('english'))]
    review = ' '.join(review)
    corpus.append(review)

Creating Bag of Words Model

For this step we use the CountVectorizer class from sklearn.feature_extraction.text.

from sklearn.feature_extraction.text import CountVectorizer
cv = CountVectorizer(max_features = 1500)
X = cv.fit_transform(corpus).toarray()
print(X)

[[0 0 0 ... 0 0 0]
 [0 0 0 ... 1 1 0]
 [0 1 0 ... 0 0 0]
 ...
 [0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]]

Summary

In this blog post we see how to implement bag of words technique to extract numeric features from cleaned and preprocessed textual data. In the next blog post we will learn about TF_IDF technique to extract numeric features from cleaned and preprocessed textual data.