I have a dataset in which each sample has only two features. I designed my own gradient descent algorithm, and applied it to my dataset. However, I could not obtain a result. Then, I printed the parameters generated during gradient descent, and I realized that zigzagging happens. In other words, the algorithm cannot converge to optimum point due to the value ranges that my features vary in. My features have very different value ranges.

Then, I performed mean normalization operation for feature scaling to my dataset and trained my data. My gradient descent algorithm worked perfectly well. It converged very quickly. After that, I trained another model by using built-in scikit LinearRegression library to check the success of my gradient descent algorithm. Both models, first of which is trained by my gradient descent and second of which is trained by sklearn.linear_model.LinearRegression, make very close predictions. As a result, my algorithm works properly.

The problematic issue is the values of parameters which are slope m and bias value b in the equation mx +b. Since I performed mean normalization to the features of my data set, feature space is changed. Hence, parameter values that I obtain at the end of my gradient descent and the parameter values that built-in acquire at the end of the training are not same.

Is it possible to track original parameters even if feature scaling is performed ?

I provided equation of mean normalization that I performed below:

$x^{new} = \frac{x - u}{max(x) - min(x)}$


You just need to always use this scalling for your futures (also form the test set) if you want to run a prediction, the value xnew will always between (0,1) so I should not be a problem for your prediction. But don't normalize your prediction value, this is not needed.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.