Implement Naive Bayes Classifier with Matlab

Estimating joint density directly from the joint density table requires intensive computation, especially with large dataset. Naive Bayes classifier was developed to simplify the problem by assuming the conditional independence of of the features given the class. To implement a Naive Bayes classifier in Matlab, two separate functions could be developed: nb_train and nb_test. The nb_train() function takes in training dataset x and y, with each row of x represents the feature vector of one training instance and the corresponding row in y contains the class label for that instance. Inside nb_train() function, data points for each class are grouped. For each class, the conditional probability of each feature is calculated. With Matlab, computation time could be greatly reduced if using matrix arithmetics rather than looping through every element. Another thing worth attention is that, smoothing is a widely employed technique in Naive Bayes algorithm implementation when calculating conditional probability. One common approach is additive smoothing, as illustrated below. This example calculates conditional probabilities of all n features for a binary class problem. catXOnes and catXZeros are the concatenated matrices for the two classes individually. The computed conditional probabilities matrices as well as class priors should be put together inside a model struct as the return value. Inside nb_test function, class label prediction is performed according to the following formula: Information retrieved from the model struct is applied on the testing data points to calculate the probability of belonging to each class. The class label is determined as the one with the highest probability.