Linked Questions
18 questions linked to/from What algorithms need feature scaling, beside from SVM?
80
votes
8
answers
119k
views
How and why do normalization and feature scaling work?
I see that lots of machine learning algorithms work better with mean cancellation and covariance equalization. For example, Neural Networks tend to converge faster, and K-Means generally gives better ...
- 1,667
69
votes
3
answers
34k
views
Variables are often adjusted (e.g. standardised) before making a model - when is this a good idea, and when is it a bad one?
In what circumstances would you want to, or not want to scale or standardize a variable prior to model fitting? And what are the advantages / disadvantages of scaling a variable?
- 5,998
41
votes
2
answers
42k
views
Dropping one of the columns when using one-hot encoding
My understanding is that in machine learning it can be a problem if your dataset has highly correlated features, as they effectively encode the same information.
Recently someone pointed out that ...
- 718
11
votes
2
answers
5k
views
Whitening/Decorrelation - why does it work?
Given some whitening transform, we change some vectors $\textbf{x}$, where features are correlated, into some vector $\textbf{y}$, where components are uncorrelated. Then we run some learning ...
- 1,086
6
votes
2
answers
3k
views
Data matrix, predictor matrix, observation matrix, model matrix, and design matrix. What do they mean?
Is there a clear distinction between these terms? To the best of my knowledge:
Suppose we have $N$ observations and $p$ predictors.
predictor matrix $\in \mathbb{R}^{N\times p}$ is synonymous to ...
user145002
5
votes
1
answer
383
views
"Joint" dummy variables for two different variables
I am supposed to show the hazard ratio (HR) stratified by gender (1= female vs. 2= male) and age groups (quartiles, 1-4)*. The combination "female" and "first quartile of age" is supposed to be the ...
user213325
5
votes
1
answer
11k
views
If you standardize X, must you always standardize y?
Related reading:
When conducting multiple regression, when should you center your predictor variables & when should you standardize them?
When and how to use standardized explanatory variables in ...
- 792
4
votes
3
answers
3k
views
Why does scaling the features affect the prediction of a regression?
I'm working on a regression problem using the support vector regression model from sklearn and using MinMax to scale the features, but by using it I get a different result for the regression, does ...
4
votes
1
answer
5k
views
Does XGBoost require standarized data?
In related question (What algorithms need feature scaling, beside from SVM?) every answer stated that XGBoost doesn't require any standarization, but someone wrote in comment that:
+1. Just note that ...
- 247
4
votes
0
answers
138
views
When to scale or standardize data in regression [duplicate]
Many statistical software ask whether to standardize data or no:
What is a general rule to when data should be standardized?
Do we standardize categorical variables?
Is there a difference in how ...
- 2,144
3
votes
0
answers
107
views
When not to use standardization on variables [closed]
Can someone give me a counter example of when we should not use standardization on variables? I understand what standardization is but i am not getting the point why we need to standardize variables? ...
- 225
2
votes
0
answers
117
views
Relation between Range of feature and Range of parameters in Logistic Regression?
It may be a very basic question but as a beginner in MachineLearning I still cannot figure out the answer.
In Andrew Ng's machine learning course he explained why we need feature scaling in http://...
- 31
2
votes
2
answers
3k
views
Do we need to standardize when our data is univariate?
In this question: What algorithms need feature scaling, beside from SVM?
it is said that we need to standardize so that all features are weighted equally.
But what if we only have as features: time ...
- 1,159
1
vote
1
answer
553
views
Should you scale the dataset (normalization or standardization) for a simple multiple logistic regression model?
I have read a lot of conflicting literature about scaling the dataset (using methods such as normalization or standardization) for a multiple logistic regression model, and I am wondering if scaling ...
- 13
1
vote
1
answer
3k
views
Multiple regression of variables with different units
I'm new in statistical modelling and using R, so please excuse my mistake for this question.
I want to make multiple regression model with these variables:
Revenue (in million USD) as dependent ...
- 13