Ways to improve the accuracy of a Naive Bayes Classifier?

user_419126 0
Viewed 52982

I am using a Naive Bayes Classifier to categorize several thousand documents into 30 different categories. I have implemented a Naive Bayes Classifier, and with some feature selection (mostly filtering useless words), I've gotten about a 30% test accuracy, with 45% training accuracy. This is significantly better than random, but I want it to be better.

I've tried implementing AdaBoost with NB, but it does not appear to give appreciably better results (the literature seems split on this, some papers say AdaBoost with NB doesn't give better results, others do). Do you know of any other extensions to NB that may possibly give better accuracy?

naivebayes machine-learning
6 Answers

Improves Naive Bayes classifier for general cases

  1. Take the logarithm of your probabilities as input features

We change the probability space to log probability space since we calculate the probability by multiplying probabilities and the result will be very small. when we change to log probability features, we can tackle the under-runs problem. Formula of predicted y

  1. Remove correlated features.

Naive Byes works based on the assumption of independence when we have a correlation between features which means one feature depends on others then our assumption will fail. More about correlation can be found here

  1. Work with enough data not the huge data

naive Bayes require less data than logistic regression since it only needs data to understand the probabilistic relationship of each attribute in isolation with the output variable, not the interactions.

  1. Check zero frequency error

If the test data set has zero frequency issue, apply smoothing techniques “Laplace Correction” to predict the class of test data set.

More than this is well described in the following posts Please refer below posts.

  1. machinelearningmastery site post
  2. Analyticvidhya site post
user_128944680
Related