Problems with Gradient Descent

• important general paradigm when
  1. continuously parameterized hypothesis
  2. the error can be differentiated with respect to the hypothesis parameters
• The key practical problems are:
  1. converging to a local minimum can be quite slow
  2. if there are multiple local minima, then there is no guarantee that the procedure will find the global minimum (Notice: The gradient descent algorithm can work with other error definitions and will not have a global minimum. If we use the sum of squares error, this is not a problem.)