Problems with McNemar

• did not measure variability due to choice of training set or internal randomness - only use if we believe these to be small
• it does not directly compare performance on training sets of size but only - we must assume the relative difference will still hold for training sets of size