Ensembles of Classifiers

• An ensemble of classifiers is a set of classifiers whose individual decisions are combined in some way (typically weighted or unweighted voting) to classify new examples.
• Ensembles are often much more accurate that the individual classifiers that make them up.
• An ensemble can only be more accurate that its component classifiers if the individual classifiers disagree with one another.
• If individual hypotheses make uncorrelated errors at rates exceeding 0.5, then the error rate of the voted ensemble increases.
• Key: individual classifiers with error rates below 0.5 whose errors are at least somewhat uncorrelated