Maximum Likelihood & Least-Squared Error

• Bayesian justification for many ANN and other curve fitting methods
• Probability density: (necessary for continuous target)
• the random noise variable, , is generated by a Normal probability distribution (i.e., smooth bell-shaped distribution that can be completely characterized by its mean, , and standard deviation, )
• , where random noise is drawn independently from a Normal distribution with 0 mean
• maximum likelihood hypothesis might not be MAP hypothesis, but if one assume uniform prior probabilities over the hypothesis then it is
• Does noise have normal distribution - noise generated by the sum of very many independent identically distributed random variables will be Normally distributed regardless of the distributions of the individual variables
• in reality different components to noise probably don't have identical distributions
• above analysis only allows noise in the target - for instance when predicting weight based on height and age analysis assumes noise in the measurement of weight but perfect measurements of height and age - analysis is significantly more complicated if this simplifying assumption is removed