Combining Classifiers

• 1) unweighted vote - bagging, ECOC - shown to be robust
• class- probability estimate if classifier can produce class probability estimates
• 2) many weighted voting methods - for regression weight should be inversely proportional to the variance of the estimates of h
• classification - weights proportional to the accuracies
• 3) learn good weights - gating function or gating network - overfitting a problem
• 4) stacking - use outputs of L classifiers as attributes for target in leave one out - good results in combining different forms of linear regression