Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [standardization]

Usually refers to "z-standardization" which is shifting and rescaling data to assure they have zero mean and unit variance. Other "standardizations" are possible, too.

0
votes
0answers
7 views

Standardization by reference group [on hold]

I am trying to standardize my variables in SPSS, but I cannot find the correct procedure to calculate it correctly. I have a clinical and a non-clinical group, and my goal is to standardize the scores ...
-1
votes
1answer
62 views

Normalizing input data for Machine Learning model ruins everything

I am trying to build a Random Forests model for regression purposes. I am using a very standard pipeline: a initialize the model (from Scikit-learn), I fit the model with ...
1
vote
3answers
42 views

Standardised mean absolute error (SMAE) and how to calculate it?

I am using the mean absolute error mean(abs(obs - pred)) as one of the measures assessing the fit of my model. I would also like to have a standardised measure ...
0
votes
0answers
13 views

Z score standardisation vs min-max scaling for feature selection

I am applying l1 norm on the input weights of a single layer MLP. I wanted to know if I should standardize or min-max scale ([0 1] feature scaling) my input data?
0
votes
0answers
6 views

Construct least-squares ultrametric (hierarchical clustering) fits to doubly-standardized “flow” tables and compare to single-linkage-type fits

Figure 1 of the paper, "Hierarchical Migration Regions of France" (IEEE Transactions on Systems, Man and Cybernetics, 4 (1976) 321-324) (https://www.researchgate.net/publication/...
0
votes
1answer
33 views

Standardization of Data

I have a dataset which consists of Sales for Product1 and Product2. It also tells if the <...
0
votes
0answers
7 views

Direct standardisation with missing values on ages

I am aiming to perform a direct standardisation to calculate the prevalence rate of a certain condition in an area. What I did was to merge different routinely collected health data to be able to ...
0
votes
0answers
12 views

Calculating ICC using z-scored standardized or unstandardized DV in Multilevel Linear Model?

I am doing multilevel linear modelling and I am calculating my ICC for my random intercept model. However, when I use the z-scored standardized DV (reaction time), the intercept and residual variance ...
1
vote
1answer
12 views

Should I standardize all variables before a PCA separately if some share the same units

I have a matrix that contains >2000 variables which can be divided in 4 groups of ~500 variables with each group having a distinct unit. I need to standardize the matrix before running a PCA, but when ...
0
votes
0answers
10 views

Group mean&SD with respect to which z-scores are computed

I have a research report that gives the standardised test scores from a number of subjects. However, it is not specified with respect to which group the z-scoring was made. Thus, for each subject ...
1
vote
0answers
19 views

Questions about standard error

We are trying to create a table of standardised effects and standardised errors to compute for a meta-analysis. And I had a few questions around this Can you get a standard error for pearson ...
1
vote
1answer
16 views

Un-standardize feature weights

I have a linear regression model $y_a = \theta_a^T\tilde{f}$, where $\theta_a$ is a vector of learned feature weights and $\tilde{f}$ is my standardised feature vector; $$ \tilde{f} = \frac{f - \mu}{\...
1
vote
2answers
34 views

Why z-score/standard score is a linear transformation

Can someone please help me understand why the standard score $(X - \mu)/\sigma$ is a linear transformation since both mean and standard deviation depend on X?
0
votes
0answers
16 views

Centering / standardizing leads to very different results for GLM (logistic, poisson, negative binomial distribution)

I have a dataset with count data and around 1 million observations. My regressions contain around 40 variables (binary and continuous) and 10 thousand fixed effects. I analyze this dataset with linear,...
6
votes
2answers
1k views

What is the reasoning behind standardization (dividing by standard deviation)?

Why does dividing a dataset by sigma make the sample variance equal to 1? Assuming a zero mean for simplicity. What's the intuition behind this? Dividing by the range (max-min) makes intuitive sense....
0
votes
0answers
5 views

MIMIC Model and standardization

I'm running a MIMIC model in MPlus with a dummy coded covariates and binary manifest variables. Which standardization should I use to calculate the ETS effect size for DIF, std, stdy, or stdyx?
0
votes
0answers
16 views

How to perform feature scaling on noise removel process?

i'm working on dataset contain machinery sensor data. each column(feature) represent different sensor data(pressure, temperature, speed, etc) of the machine part. here task is to predict normal ...
0
votes
1answer
74 views

how does Keras ImageDataGenerator standardize data?

If I understand correctly the ImageDataGenerator class is a generator and returns batches of images when called, but what I don't seem to understand is: featurewise_center ...
0
votes
0answers
12 views

What is the relationship between the standardized multiple regression coefficient & the semi partial correlation for models with k>2 predictors?

I have found myself Googling this question more than once: ¿What is the relationship between the standardized multiple regression coefficient (the standardized partial slope) and the corresponding ...
0
votes
1answer
18 views

z-score of one variable, but for two groups

I need to transform the score of a variable to z-scores, because of two different scales that were used according to the subjects' age. However, there are two groups: control and diseased. Do I ...
0
votes
0answers
41 views

Data normalization for recommender system

Does anyone know whether it's a good idea to standardize your data by replacing it with the percentile which it occupies in a distribution? Instead of substracting the mean and dividing by standard ...
0
votes
0answers
9 views

Why are standardised coefficients from STB not equivalent to standardised input data?

I recently saw a question on cross-validated which I wanted to double-check. The statement said that the following two outputs should be equal: Running a multivariate regression on non-standardised ...
0
votes
2answers
23 views

Sample Mean expressed using Standard Normal Distribution

Let $\bar{X}_n = X_1 + \dots X_n$ where $X_i \sim N(0,1)$. We can easily verify that $\bar{X}_n \sim N(0, 1/n)$. Thus $\text{Var}(\bar{X}_n) = 1/n$. Let the density of $X \sim N(0,1)$ be denoted $\...
0
votes
0answers
16 views

Facilitating Comparisons - 1 SD difference

I came across a paper that mentions this in their statistical analysis: To facilitate comparisons among variables, the hazard rate ratio (HRR) was calculated for a 1-SD difference for each of the ...
1
vote
0answers
15 views

k-means and (re?) standardisation of a sub-set

I have data which is customer purchases of items in each of three months: I have summed the data over the three months for each customer; calculated the proportion of purchases that each item ...
1
vote
1answer
43 views

Multiple regression of variables with different units

I'm new in statistical modelling and using R, so please excuse my mistake for this question. I want to make multiple regression model with these variables: Revenue (in million USD) as dependent ...
0
votes
0answers
19 views

Setting standard deviation to 1 in PCA

I was going through Introduction to Statistical Learning in R, Seventh Printing, and I have a question about Principal Components Analysis. If the first principal component is measured as the ...
0
votes
0answers
23 views

Can standardized variable be used to generalize sample results to whole population?

I have a dependent variable(wage), and two variables, that are correlated to wage (country and ...
2
votes
1answer
37 views

$L_2$ norm of product of two vectors

Let's assume we have two matrices $A^{d\times 1}$ and $B^{1 \times e}$, and we define their product as $C^{d\times e}$. Assuming $A,B$ are real valued with all entries in $[-1,1]$. I can intuitively ...
0
votes
0answers
12 views

Standardisation (normalising) for feature scaling in non-logged data for within transformation model

I wonder if someone could provide an insight for my case. I am interested in normalising my data to obtain unitless state that will allow me to use them in a quadratic model (not translog). Based on ...
2
votes
1answer
27 views

accounting for age effects: two age groups and within-group age differences

I am analyzing data from a study with two groups of about 25 healthy participants each, one young (mean age ~25 [~18-~30] years) and one older (mean age 59 [45-75] years). The study was set up mainly ...
0
votes
0answers
25 views

Standardizing variables with negative values? [duplicate]

This question here Standardized VS centered variables says you can only standardize or center variables after transforming them into log. Because my variables are in different units, my coefficients ...
2
votes
2answers
74 views

Standardizing variables with negative values in R

Because my dependent variables in a multiple linear regression model are in different units, some coefficients are > 10,000 and others less than 0. I need to standardize them by using the ...
0
votes
0answers
9 views

Usage of correlated (non-linear correlation) variables in an experiment and standardisation of variables values

The setting (see the dataset at the end of the question) The setting of the problem is this: I ask many multiple choice questions; only 1 out of the 3 available answers is correct; I ask the same ...
3
votes
0answers
32 views

Should I standardize my variable for regression before nonlinear feature transformation?

I would like to fit a non-linear model by doing nonlinear feature transformation first (e.g. exp, log) and then using linear regression (or regularized linear regression). However, I am stuck at ...
0
votes
1answer
51 views

Should I standardize mixed variables before lasso or pca?

I have a large data of both binary, categorical and continuous variables. I would like to do the lasso to select the variables for regression. Besides, i also want to do pca to reduce the dimension. ...
0
votes
0answers
16 views

Standardising input for regression - higher order terms

I want to standardise my data for use in a regression. Taking my variable h and following Gelman 2007, I subtract the mean and divide by two standard deviations, ...
1
vote
1answer
57 views

How to to get normal distributed neural network output [closed]

I am trying to build a neural network that predicts a pair of latitude / longitude coordinates following a previous pair of latitude and longitude (highly simplified). The latitudes and longitudes ...
0
votes
1answer
30 views

How to normalize Medical data to further use in Electronic Health Records? [closed]

How can we normalize medical data extracted from patients to be used later in Electronic Health Records? Example for data: Age, temperature, time, blood test. I have come across several papers that ...
0
votes
0answers
14 views

Does it Matter if Categorical Features are Standardized?

I have a dataset I am scaling in order to run a machine learning algorithm. I know it's bad practice to standardize binary features in regression, but I was wondering if it actually makes a difference ...
0
votes
0answers
20 views

What's the rule of thumb for when to normalize and when to standardize? [duplicate]

I'm reading about normalizing data for predictive analysis. on Statistics how to, they write (https://www.statisticshowto.datasciencecentral.com/normalized/) Normalization usually means to scale a ...
0
votes
0answers
29 views

Value at Risk VaR with monthly return

The picture below shows the problem. I have tried to solve in this way: 0,001*1,88+2400*0,002 by following a formula in theory slides. But obviously there is a problem in that. Could someone help me ? ...
0
votes
0answers
40 views

PCA: should standardization be applied on features or samples?

I am struggling a little bit with PCA. I understand that standardization is an important part of the algorithm but I do not understand which elements should be standardized. Let's say I have a 10x100 ...
3
votes
0answers
40 views

Standardizing qualitative variables in R to perform glm's, glm.nb's and lm's [closed]

I want to standardize the variables of a biological dataset. I need to run glm's, glm.nb's and lm's using different response variables but the same explanatory variables. The dataset contains counts ...
0
votes
0answers
26 views

Transforming raw scores to scaled scores using a proprietary scale

I'm working with questionaire answers from the Child Behavior Checklist (CBCL) for my master's thesis. The 113 items on the questionaire are organized into several subscales for different areas of ...
2
votes
1answer
28 views

Standard population for SMR estimation

I have deaths by counties from 1980-2010 by age and sex. I want to calculate the SMR and then map the rates. My question is what standard population should I consider. If the SMR is Yi (# deaths ...
0
votes
0answers
11 views

How to choose standardisation method

I am working on an analysis in which I have been asked by colleagues to supply "standardised" coefficients, by which they mean on a comparable scale for ease of interpretation (I am aware that it ...
0
votes
1answer
21 views

Z Score Values Used and Interpretation

I have been working on a time series-like data set from using calcium indicators (waveform data). A script I was given transforms the voltages (x values) into Z-scores. However, the mean and standard ...
1
vote
1answer
50 views

GLM standardisation with quadratic terms

I had understood that using linear transformations, such as centering and scaling, of predictor variables in GLMs does not affect the t/z-values, and thus nor the p-values (except for the intercept). ...
0
votes
0answers
31 views

Calculation of standard deviations for regression coefficient standardization when intercept is not present

I took the code of function lm.beta::lm.beta in R and wrote it in this way to be more understandable (and commented the lines I'm not interested in): ...