Recently many researchers have investigated various techniques combining the predictions from multiple predictors to produce a single predictor. The resulting predictor is generally more accurate than an individual one. The ensemble of predictors is often called a committee machine (or mixture of experts). In a committeemachine, an ensemble of predictors (often referred as a weak learning machine or simply a machine) is generated by means of a learning process; the overall predictions of the committee machine are the combination of the individual committee members’ predictions. Figure 1 presents the basic scheme of a committeemachine. Eachmachine (1 through T) is trained using training examples sampled from the given training set. A filter is employed when differentmachines are to be fedwith different subsets (denoted as type A) of the training set; in this case, the machines can be run in parallel. Flows of type B appear when machines pass unclassified data subsets to the subsequent machines, thus making a hierarchical committeemachine. The individual outputs yi for each example are combined to produce the overall output y of the ensemble. Bagging and boosting are the two popular committeemachines that combine the outputs from different predictors to improve overall accuracy. Several studies of boosting and bagging in classification have demonstrated that these techniques are generally more accurate than the individual classifiers.
|  |
|
Research >
Committee machines |
Read more about review of boosting algorithm in the refereneces below.