# How does having different scales on features make an elliptical contour plot? [closed]

I have been taking Andrew Ng's Machine Learning course, and in the lesson on feature scaling's effect on gradient descent, I just can't understand how because of the different scales on the features cause an elliptical contour plot of the cost function. I made the inference from the contour plot that the cost function increases and decreases very rapidly with an increase of decrease in theta 1, and it increases or decreases very slowly with an increase or decrease in theta 2. (X1=Sq feet of the house. X2=Number of bedrooms.) But I still cant understand how having different ranges fuels that. Please help

• Please explain the senses in which you use "scales" and "ranges" and how these terms are related (you seem to use them interchangeably, but that's only a perception). In particular, what mathematical role would either of these concepts play in establishing a cost function?
– whuber
Commented May 10 at 14:12

We can use two functions here as examples to help to understand the concept of derivative.

$$f_1 (x_1, x_2) =x_1^2+x_2^2$$

$$f_2 (x_1, x_2) =x_1^2+10x_2^2$$

If we plot the contour we will find $$f_2$$ is elliptical. The reason is that along the $$x_2$$ direction, we can reduce to cost more quickly than $$x_1$$ direction.

For example, if we are at point $$(1,1)$$, $$f_1(1,1)=2$$. For next point, $$f_1(0.9,1)=f_1(1,0.9)=1.81$$, i.e., they have the same value.

But for $$f_2$$, it is a different story. $$f_2(1,1)=11$$, $$f_2(0.9,1)=10.81$$ and $$f_2(1,0.9)=9.1$$. So, we can see, if we reduce $$x_2$$ it is more effective to reduce the cost.

This is why it make make the elliptical contour.

• Ok, so according to you, if x2 is of large value, then we can reduce to cost function more faster, but it makes the elliptical contour? Commented May 12, 2020 at 10:56
• But my professor says that θ will descend quickly on small ranges and slowly on large ranges, which is the opposite of what you said, please clarify. Commented May 12, 2020 at 10:57