Question 6

• Given a learning algorithm and a small dataset , what is the accuracy of the classifiers produced by when is trained on new training sets of the same size as ?
• Kohavi shows that stratified 10-fold cross validation produces fairly good estimates.
• Note in any resampling approach we cannot train on training sets of exactly the same size. We train on smaller datasets (90% the size of ) and rely on the assumption that the performance of learning algorithms changes smoothly with changes in the size of the training data. This assumption is not always valid.