Shifting and rescaling data to assure zero mean and unit variance.

learn more… | top users | synonyms (1)

1
vote
0answers
20 views

Normalization/Standarization for Clustering visualization

I'm performing visualization of a dataset clustered with k-means. I compute a weight for each cluster and I draw a circle as big as its weight. But it seems like after the clustering some values are ...
1
vote
0answers
20 views

Should standardization be done using leave-one-out?

When we have data from a normal distribution, we may wish to standardize the values in our sample to $N(0,1)$. In such case it is customary to divide each observation by the sample mean and standard ...
0
votes
0answers
13 views

Glmnet standardize not consistent with prior standardization

I am running glmnet with standardize=TRUE. Now I try to pre-standardize my data, and un-standardize after the regression. But the results are inconsistent: ...
0
votes
1answer
18 views

How to apply Box Cox to train and test data?

I am trying to standardize my data to performing prediction on it. Some of the features in my data are skewed and hence I am applying Box Cox transformation to reduce skewness. My data also ...
0
votes
2answers
46 views

Combining two time series with normalization

I have two sets of time series data with different distributions. Basically, one is count of likes for a post and the other one is the likes on the corresponding comments. It looks like this Now, I ...
0
votes
1answer
14 views

create indices by standardising variables

I have 257 variables, which influence behaviour altruism of humans. They can be assigned to the 3 following criteria: internal, external or selection. I check the influence of the individual variable ...
0
votes
0answers
10 views

Is it accepted to fit model with standardized data and predict on non-standardized data? [duplicate]

If you standardize your training data, then can it work on unstandardized data during predictions accurately? Many algorithms require the feature data to be standardized and I am wondering how/why/...
0
votes
2answers
45 views

standardising non normally distributed predictors for regression

In regression, standardization is recommended in ordered to assess the relative importance of predictors. However there seems to be an assumption of normality? How would the interpretation work for ...
0
votes
3answers
48 views

Standardizing some features in K-Means

I have 21 features in my dataset, some features are more important than others. As a fact I know, if I don't standardize (mean=0, SD=1) any features, then features with low variance will have slightly ...
0
votes
1answer
21 views

Normalising/Standardising Data for Machine Learning

If we have a system in which we normalise and standardise the data, I'm interested into how to most effectively do this. For a test system we can apply normalisation and standardisation techniques to ...
0
votes
1answer
46 views

What does this statement “unit L2 norm” mean?

While I am reading the lars paper I encountered this statement " Note that these data are first standardized to have zero mean and unit L2 norm before they are used in the examples." What does it ...
0
votes
0answers
16 views

Do I standardize the response value as well? [duplicate]

In linear regression when my variables have a different scale do I have to standardise only the independent variables or the response as well?
6
votes
3answers
226 views

Can I test for correlation between variables before standardize them?

What I want to do is to construct GLMM's to evaluate resource selection, and I have a set of variables (some representing distances and others representing % of land cover). Can I test for ...
0
votes
0answers
13 views

Question regarding variables transformation for GLMM's

I have a response variable (binary:1/0) and a set of explanatory variables, with different units: some have values in %, others in meters (distances, altitude and differences between elevation pixels),...
1
vote
0answers
10 views

(Psychology research) Is it appropriate or coherent to standardize these variables?

I am doing a project where I'm analyzing various lexical dimensions of speech from folktales from 10 countries. And the program, LIWC, returns a value for each category with that value representing ...
0
votes
0answers
18 views

z-score normalisation - how to achieve this for score combination?

I have read that to perform a score fusion from two different classifiers on two different datasets then the score must be normalised. I understand I can use z-score normalisation / max-min ...
2
votes
1answer
42 views

Reasons NOT to use standardised data in multivariate analysis

Question: Can you give any reasons/examples when it is more appropriate NOT to standardise continuous metric independent variables when performing multivariate analysis? Background: I am an undergrad ...
0
votes
0answers
61 views

Data normalization/standarization and comparing

I have a question related to data comparing. First of all, my dataset is composed by cities of the world. In this cities, we have a maximum of 24 tags that indicate what these cities are best for. ...
0
votes
1answer
46 views

How to standarise percent values 0 - 100%

I'm working on dataset that contains a variable from 0 to 1 (0 - 100%). Distribution of the variable differs depending on the context (defined by another variable). Depending on the context the ...
0
votes
0answers
12 views

normalization and rescaling definitions are mixed up?

in: https://en.wikipedia.org/wiki/Normalization_(statistics) it's written that rescaling(min_max_scaling) and standarization are types of Normalization what i see in alot of stackoverflow answers is ...
0
votes
1answer
25 views

Standardization and prediction on new data

As far as I know it is common practice to do standardization of variables before shrinkage or PCA, which are methods I intend to use on my model selection for a predictive model. But the problem is, ...
0
votes
0answers
23 views

Heteroskedasticity consistent SEs with Proc ROBUSTREG in SAS

We are using the Proc ROBUSTREG command in SAS to down-weight the influence of the outliers (mainly in the Y-direction). Furthermore, we wish to use heteroskedasticity consistent SEs and standardized ...
0
votes
1answer
20 views

Ranking submissions based on multiple judges

I have a situation in which we have 20 proposals and five judges. Each proposal is reviewed by three of the five judges, assigned to the proposal at random. Each judge scores the proposal out of ...
0
votes
1answer
70 views

Standardization in neural network online training

It is common knowledge that the inputs to a neural network should be standardized to have mean 0 and variance 1 (see this thread for example, or the LeCun paper). And as long as one does batch ...
4
votes
1answer
55 views

Use z-scores to determine the best strategy for airlines

Most airlines board passengers starting from the back of the plane and then working their way towards the front (after boarding priority classes and passengers). In an episode of Mythbusters, Adam ...
5
votes
1answer
99 views

Should the correlation PCA projection be computed on original or normalized samples?

Suppose we compute the correlation PCA of a dataset $X$ (with $m$ variables and $n$ observations) by first normalizing the input variables. That is: mean -> 0 and standard deviation -> 1. Let us ...
4
votes
0answers
30 views

When to scale or standardize data in regression

Many statistical software ask whether to standardize data or no: What is a general rule to when data should be standardized? Do we standardize categorical variables? Is there a difference in how ...
0
votes
1answer
49 views

Why doesn't standardization work in the linear regression?

I have a matrix containing the attributes of the item and their corresponding rating. All of the attributes are in the range of (0,1) and the rating is in [1,5]. I transform the range of rating to (0,...
2
votes
1answer
49 views

Variable standardization / scaling for PCA when all dimensions already have same scale [duplicate]

Often when PCA is performed on exam results where all variables (dimensions) have the same $0$ to $100$ scale, scaling is none the less applied. For different scales I can see the purpose of it, but ...
1
vote
0answers
28 views

What is the meaning of a t-statistic?

When working with a normal distribution, the z-score can be interpreted as the number of standard deviations from the mean a given value is. ($z=2$ means that $ x $ is 2 standard deviations from the ...
0
votes
0answers
25 views

Using trimmed means and Winsorized variances to compute standardisation of data

I am looking into the pros and cons of each normalisation technique for work and it got me thinking. What if I used trimmed means and the sqrt of Winsorized variances to compute the standardised data? ...
0
votes
0answers
18 views

Rescaling vs Standardization of features

Is there any general rule of thumb or any justified rule to choose whether to scale a dataset using Rescaling (for each feature, subtract the min value and divid by the max - min) or Standardization (...
1
vote
0answers
12 views

Comparison of regression coefficients between (nested) geographic regions

I would like to compare the effect of an explanatory variable (say X1) on a response variable (Say Y) between two geographic regions in which one is a sub-region of the other. For example, I am ...
1
vote
1answer
58 views

Standardized LASSO in R still has intercept

I understand the need to standardize variables when performing LASSO in R (I'm specifically using cv.glmnet, and setting ...
0
votes
1answer
33 views

Normalize all data before cross-validation or normalize every train part separately and use same properties for test part?

Suppose that we want use 5-fold cross-validation for a support vector regression(SVR) model. We should normalize total data before cross-validation process or we need normalize every train part ...
1
vote
0answers
28 views

What is the correct way of standardizing data when there are training, validation and test set [duplicate]

When standardizing data before training a neural network, say by subtracting the mean and then dividing by the standard deviation for each variable, there are several ways one could go about that and ...
0
votes
1answer
20 views

Best way to examine longitudinal data?

I had 20 patients come to clinic once a month for 6 months. At each visit we collected baseline data. We then gave the patients 3 different treatments to see the effects for each visit. Thus we have ...
1
vote
1answer
121 views

Normalization of count data of time periods with different length

I have count-data from two time-periods which differ in length. The event I'm counting is in both periods the same kind of event. Period 1 is 120 hours Period 2 is 48 hours At the end I have ...
7
votes
1answer
112 views

Is it a mistaken idea to use standardized coefficients to assess the relative importance of regression predictors?

There are various questions that speak to the relative merits of various methods of assessing the importance of regression predictors, for example this one. I noticed that in this comment @gung ...
0
votes
0answers
84 views

Standardization with mean/std or median/IQR?

I have a dataset with 10000 data points and 20 features. The features are not normally distributed (most of them have a generalized extreme value or burr distribution and all values are greater or ...
0
votes
1answer
81 views

Standardization before PCA with data in same units and similar interval? [duplicate]

We have 16 variables which are indices produced by calculations based on ratio (unitless in fact). Some examples of the ranges of our variables are (0.450-0.750), (0.000 - 0.800) and (0.000 - 1.000). ...
0
votes
1answer
51 views

Transformation of a Skewed Composite Outcome made up of 2 Z-scores?

I am running a repeated measures mixed model. For my outcome variable, I would like to sum 2 continuous variables, which consequently are both Z standardized in order to do so. However, my outcome ...
0
votes
1answer
53 views

Normalization vs Standardization for multivariate time-series

I'm using DTW as a distance measure for comparing two multivariate time-series. I want to be able to cluster data using DTW as distance measure, since time-series may be shifted, skewed. Since there ...
0
votes
0answers
41 views

How to standardise linear regression

I have a set of death rates (that range from about 0.1 to 0.5), a set of body weights (that range from about 2 to 80), and I want to calculate standardised residuals for the death rates after ...
2
votes
0answers
193 views

double feature value in ridge regression, coefficients change?

In ridge regression using unnormalized features, if you double the value of a given feature A (i.e., a specific column of the feature matrix), what happens to the estimated coefficients for every ...
0
votes
0answers
16 views

Method of standardization of essay grades from a number of graders

Basically, there is a group of us grading a few hundred essays. We divided the number up so each of us grade 50, assigning a score 0-100 given a rubric. However, there is still subjectivity in the ...
0
votes
0answers
22 views

Standardizing Continuous Predictors in a Model While Not Standardizing Categorical Predictors

I have a logistic regression model where I have predictors that are categorical (binary) and continuous. It makes sense for me to standardize my continuous predictors, as I am doing something similar ...
0
votes
0answers
17 views

Standardization of all variables and weighting of some variables for clustering

I am trying to segment a database based on certain variables. I understand that before i do start clustering, i should standardize all the variables. This can be done by Z score or other methods. ...
0
votes
0answers
40 views

Is z score transformation necessary before running a mean comparison statistic?

I have 5 sets of scores from 5 different tests given to some students (let’s say group A) across 5 sessions (i.e. within-subject design). The first test (i.e. pretest) has 33 items, the last (i.e. ...
1
vote
0answers
47 views

Centering and scaling skewed distributions

I have a dataset where the features are skewed (non normal) distributions. My preprocessing pipeline consists of the following steps: Missing values imputation Centering and scaling (zero mean and ...