Naïve Bayes classifier evaluates conditional probability of each class given the observations. Assumes conditional independence between features. Bayesian Belief Networks evaluate joint probability of all variables given the observations. Take into account statistical dependence between features according to the edges in the dependency graph and Markov blanket. All features in the training set are known in advance and used to fill probability tables at each node of the Bayesian network.
Sample application: text classification based on bag of words.
How credible are our classifications and predictions? Techniques for evaluating the quality of classifiers. Holdout estimation. Cross-validation. Bootstrap. Predicting performance of a classifier. Comparing classifiers. ROC curves.
Sample problem: finding best classifier for improving efficiency of mail promotion campaign.
Neural net is a computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs. The units generate output based on weights, and the network can learn weights from data. Multi-layer networks employ non-linearity between layers, and that allows to learn weights for non-linear models.
Sample problems: handwriting recognition, breast cancer diagnosis.
When we build models from data, we sometimes overfit and fail to extract a general model which will work well on unknown future data. Methods for increasing ability of Machine Learning models to generalize. Regularization. Special regularization techniques for neural networks.