0
$\begingroup$

This question already has an answer here:

I've read this discussion, and I'm struggling with understanding the idea behind this.

I'm following Andrew Ng's class for Machine Learning, and in one of the lectures he talks about how scaling the features can improve the convergence time of gradient descent.
Specifically:

feature scaling and gradient descent

I don't understand the intuition behind why the plot of the loss function $J(\theta)$ looks skewed (left picture).

  1. Since there is no numbering of the $x$ and $y$ axis, is the range of $\theta_1$ and $\theta_2$ values in the left plot the same?
    If they are, would that be interpreted as: small changes in $\theta_1$ would result in high fluctuations in the loss function $J$; and small changes in $\theta_2$ would result in little to no change in the value of $J$?

  2. Why does the form of the loss function change when we scale the features? Do we have to scale the target value as well? How does this reflect the plot of the data points (the "real" data from the dataset) - ?

Any 3d visualizations (2-dimensional feature vectors) out there would help a lot, I couldn't find any. Thank you.

$\endgroup$

marked as duplicate by Reinstate Monica, Michael Chernick, Siong Thye Goh, Frans Rodenburg, kjetil b halvorsen Oct 24 at 13:34

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.