Rotten Tomatoes Reviews

I have build a model to predict the freshness of the reviews from the given rotten-tomatoes-reviews dataset on Kaggle, rotten-tomatoes-reviews dataset. Rotten Tomatoes Reviews Dataest has been a popular area of research in Sentiment Analysis Area. The ability to predict and classify reviews from Rotten Tomatoes which can have a wide range of applications, including Sentiment Analysis, Social Media Monitoring, and Product Analysis. In this post, we built a Classification model using Naive Bayes Classifier given in sklearn library, a popular Classification model.

First Step:

We generate an api to access the kaggle Dataset in google colab, it will give us a kaggle.json file which we have to upload in our notebook files menu. We download the required dataset from kaggle to train our model. Our Dataset has reviews with two different labels such as Fresh and Rotten. We just unzip the downloaded Dataset in google colab, then access it from files, which you can see it here and there is one another way in which we can upload the file to our google drive and then access it from there by giving the notebook file access to your google dataset.

Second Step:

Now, we use the Python libraries to load and preprocess the data. We will use sklearn, a scientific tool-kit library, to build a Review Analysis model from the dataset. We will also split the dataset in train and test set. We used CountVectorizer function to find out frequencies of the words in our dataset.

We understand our Data and convert it into a Dataframe using the pandas library. Also we have defined a Function to evaluate the validation score for our data split, Also we have defined another function to determine the likelihood of the predicted answer.

We used a Multinomial Naive Bayes Classifier model given in the sklearn library. We fine tuned our hyperparameters by using KFold Cross Validation from sklearn, and found out the best learning rate and min_df variable which is used to determine the freqency of words that should be removed.

After initializing the model and preprocessing the data, we fit the data in our model. And train it with validation set as well. The Multinomial Naive Bayes Classifier model works by calculating the conditional probability of each feature given each class, and then using Bayes' theorem to calculate the probability of each class given the observed features in a document. The class with the highest probability is then assigned as the predicted class for the document.

Next Step:

We have check the accuracy of our classifier model after training the model. We compare the accuracy of train data and test data on our model.

Also we displayed some of the mis-predicted reviews by our model, where the resultant label was expected to be fresh and our model predicted it as rotten and vice-a-verse.

We predicted the label for one of our own review sample using our trained model. You can use many other models to do the text classification work such as, SVM, Decision Trees, Random Forest(it is just the extension of the Decision Trees), Gradient Boosting Algorithms, and Neural Networks.

Motivation:

This assignment was to understand and implement the Sentiment Analysis Model using sklearn library. The best part of building this model was to understand the data we are working on and to understand how to use the technology to understand the sentiment behind it. Sometimes the reviews could get tricky but if we build a good enough model we could successfully predict the true intentions.

Conclusion and Future Scope:

I experienced how to preprocess the text data and build a text classification model in this assignment and got to know how I can tune my hyperparameters to make my model to perform better. I have made a Google Colab notebook of it, which you can check out it is pretty easy to understand.

Try competitions on Kaggle it is fun and good for practicing your skills and keeping uptodate with current technology.