Restriction Biases and Preference Biases

• ID3 incompletely searches a complete hypothesis space from simple to complex hypotheses. Its bias is solely a consequence of the ordering of hypothesis searched. Its hypothesis space introduces no additional bias - preference of search bias
• Candidate-Elimination completely searches an incomplete hypothesis space. Its bias is solely a consequence of the expressive power of its hypothesis representation. Its search strategy introduces no additional bias - restriction or language bias
• a preference bias is more desirable
• first learner - restriction bias (linear function), preference bias (LMS algorithm for parameter tuning)