A way of re-expressing data to make their values lie within a specified range.

learn more… | top users | synonyms (1)

0
votes
0answers
8 views

normalization approach for data with high variance [closed]

There is a data set with very high variance. What are the best approaches to normalize this kind of data.
0
votes
0answers
35 views

Transformation of data for normality

I'm intended to run a linear regression model (Rain~dBZ) for my data set. I would like to know how to transform non-normal set of "Rain" column in to a normal distribution. I would really appreciate ...
0
votes
0answers
26 views

understanding Batch Normalization

I'm trying to make sense of the Batch Normalization paper, in Section 3.2, it says We could have also normalized the layer inputs u, but since u is likely the output of another nonlinearity, the ...
1
vote
0answers
26 views

What is the efficient preprocessing data in image classification task with CNN?

I am new in deep learning on image classification. I know that Machine learning algorithm are very dependent to data normalization. Usually, if we have a training data set represented with X [N*D] ...
0
votes
0answers
16 views

Using trimmed means and Winsorized variances to compute standardisation of data

I am looking into the pros and cons of each normalisation technique for work and it got me thinking. What if I used trimmed means and the sqrt of Winsorized variances to compute the standardised data? ...
1
vote
0answers
11 views

Proxies and uses for the Geometric Mean of negative (or even complex) data

I use the geometric mean (GM) as a scale factor for data normalization. To avoid the $0$ cancellation effect with positive values, I use the simple offset GM: $$ \hat{x} = ...
0
votes
0answers
14 views

Rescaling vs Standardization of features

Is there any general rule of thumb or any justified rule to choose whether to scale a dataset using Rescaling (for each feature, subtract the min value and divid by the max - min) or Standardization ...
0
votes
1answer
13 views

Normalize all data before cross-validation or normalize every train part separately and use same properties for test part?

Suppose that we want use 5-fold cross-validation for a support vector regression(SVR) model. We should normalize total data before cross-validation process or we need normalize every train part ...
0
votes
1answer
23 views

Normalising features extracted using a CNN?

I have used a pre-trained CNN to extract features from training and test images sets. The same CNN was used for all images. The CNN includes normalization layers. Before training a classifier (SVM ...
1
vote
1answer
19 views

Normalizing the Turbulence index to match other indices

I calculated the index of complexity, longitudinal entropy, and turbulence for my data. The first two indicators vary from 0 to 1, whereas turbulence varies between 1 and 16 for one dataset and ...
0
votes
1answer
17 views

How to find the unit-normalised form of the distribution?

If we have P($x_1,x_2,x_3$) $\propto$ $\delta_D$($x_1+x_2+x_3$-1) $\Theta(x_1)$$\Theta(x_2)$$\Theta(x_3)$, then how to find the form of P($x_1,x_2,x_3$)? i.e., how to calculate the integral ...
0
votes
1answer
15 views

Best way to examine longitudinal data?

I had 20 patients come to clinic once a month for 6 months. At each visit we collected baseline data. We then gave the patients 3 different treatments to see the effects for each visit. Thus we have ...
1
vote
1answer
35 views

How Do I Rank the GPA of Students from Different Colleges?

I have data from 100 students, each at a different college would like to rank them by GPA. I have the following data for each: The student's GPA: example 3.25 The all student average: 3.10 The ...
1
vote
1answer
86 views

Normalization of count data of time periods with different length

I have count-data from two time-periods which differ in length. The event I'm counting is in both periods the same kind of event. Period 1 is 120 hours Period 2 is 48 hours At the end I have ...
0
votes
1answer
22 views

MinMax normalization when all elements are same

I'm using min-max normalization to normalize time series which I compare in the following. My question is, by definition min-max normalization is defined as: ...
0
votes
1answer
12 views

Correlation of levels vs. differences vs. percents

Sometimes, I have seen people using correlation of levels, correlation of differences and also correlation of percent changes. I understand these answer different questions. For example, for "what is ...
0
votes
0answers
12 views

Scaling existing regression coefficients to predict for different dataset

I am tasked with coming up with a way to project customer activity for different groups of customers for 60 months. These groups can be based on a multitude of factors - plans, acquisition channel, ...
1
vote
0answers
24 views

How to deal with correlated response variables

I have five response variables measured at $2$ different times and I would like to know on which variables there is an improvement. I could simply divided the newest responses on the old ones and ...
0
votes
1answer
25 views

Normalization vs Standardization for multivariate time-series

I'm using DTW as a distance measure for comparing two multivariate time-series. I want to be able to cluster data using DTW as distance measure, since time-series may be shifted, skewed. Since there ...
2
votes
0answers
62 views

double feature value in ridge regression, coefficients change?

In ridge regression using unnormalized features, if you double the value of a given feature A (i.e., a specific column of the feature matrix), what happens to the estimated coefficients for every ...
0
votes
0answers
22 views

Normalize data by features or by instance?

Before applying Kernel tricks in SVM, I know I should normalize data at first. (1) But should I normalize data by features or by instances? For example, if I have n data points, each data point has d ...
2
votes
1answer
107 views

How to Normalize data?

In a study, the baseline data are recorded (blood pressure, BP1) prior to the experiment (watching a horror film). After the experiment, the data (BP2) are collected again. The problem is that not all ...
0
votes
0answers
28 views

Neural network can't learn to predict small changes

I design neural network for a prediction of numbers. I want to predict trend of a visit rate of articles. This number has big range [0,7000]. I scale numbers with min-max normalization to range ...
1
vote
0answers
53 views

Which is better to normalize data? [duplicate]

I saw on Coursera machine learning classes that is possible to normalize data in two ways: data = (data - mean) / max(data) - min(data) or you can use an Octave ...
1
vote
0answers
13 views

Normalization by z-score + range [0,1]

I am trying to normalize my dataset for further analysis. I have several data coming from different subjects, so I have first applied z-score normalization to each variable of each subject in order to ...
1
vote
1answer
23 views

Do I apply normalization per entire dataset, per input vector or per feature?

One of the ways to standardize input data for Neural network training is: \begin{equation} X = \frac{X - mean(X)}{std(X)} \end{equation} However if I have have $n$ training examples which have each ...
4
votes
2answers
93 views

When to normalize data in regression? [duplicate]

Under what circumstances should the data be normalized/standardized when building a regression model. When i asked this question to a stats major, he gave me an ambiguous answer "depends on the data". ...
0
votes
0answers
6 views

compare data from different sampling conditions

I did the same experiment on two different days. On each day I have two groups - the test and the control group. Due to changes in conditions (temperature etc.), the measurements each day came up on ...
0
votes
2answers
54 views

Use a combination of grand mean and group mean centering to standardize variables

I'm using cluster analysis to examine profiles of three variables, X1, X2, and X3. Because ...
0
votes
1answer
41 views

Normalization of values

How do I normalize a set of data? Let's say we are looking at measurements of diameter of the heart in women and men. Men will have bigger hearts, so error in measuring by the observers will always be ...
0
votes
1answer
32 views

In general, does normalization mean to normalize the samples or features?

I'm just getting into machine learning, and I have seen two conflicting practices for normalization. To be concrete, let's suppose that we have a $n \times d$ matrix containing our training data, ...
0
votes
0answers
26 views

Normalization of data between -1 and 1 [duplicate]

I would like the values -2,-1,1,2,3,4 to be normalized in the range -1 to 1. Can someone please help me with the formula to do this?
0
votes
0answers
21 views

Predicting class on data with a distribution that is different than that on which the classifier was trained

I have trained a random forest classifier on quantile-normalized data (gene expression: 20000 variables, 200 samples, RMA preprocessing). Goal: Using this classifier, I want to predict the class of ...
0
votes
1answer
24 views

Row normalization before correlation analysis for abundance data

I work with datasets in which protein abundances are reported across samples. I have some measurements of biological samples that should be more or less equal in protein abundance. After getting ...
0
votes
1answer
35 views

How to transform an arcsine distribution to a normal distribution?

I have a distribution that looks like this: U In other words small and big values are more frequent than middle values. A better graphical example of the distribution is this: Here So I have a ...
0
votes
1answer
35 views

Should the target variable be also normalised in SVM?

I understand that normalization is an important preprocessing step for using SVMs, esp. before using non-linear kernels such as the radial basis functions. Should this normalization be applied to the ...
0
votes
0answers
19 views

SVM: How to normalize |WX| > 0 into |WX| = 1

Question What are the reason/basis/rationale and the actual steps and design/mechanism behind to do the normalization to convert |WX| > 0 into |WX| = 1 in the process of getting the optimal W for SVM ...
0
votes
1answer
15 views

remove the mean over multiple measurements

I have a set of multiple measurements for each subject (i.e. each subject is assessed several days). For each set of measurements (several days of the same subject) I am calculating the mean value of ...
1
vote
1answer
57 views

Differences between two normalization approaches

I am currently try to normalize data. But I am not sure the differences between $(x - \mu)/ \sigma$ and $x/\sigma$. What are the advantages and differences of these two approaches?
0
votes
0answers
18 views

optimum degree of normalization in gradient descent

I am using steepest gradient approach for solving my my problem and I am using normalize gradient with using second parameter (I am using two parameters so for dimension match I am normalizing the ...
2
votes
0answers
63 views

Google Trends: Stitching 90 day periods of daily data together

Google Trends lets you see the amount of researches made on for a term on google during a set period of time, normalized between 0 and 100 (depending on the highest value in that period). I would like ...
0
votes
0answers
31 views

how to normalize demand/availability matrix for Citibike data

I am not a statistician but would appreciate an outside perspective on my current project analyzing citibike data. This is a bit complicated so please bear with me. My goal is to determine to what ...
0
votes
1answer
24 views

What statistical tool can be used to correct for differences in the amount of data an individual is evaluated on?

Let's say an individual gets a score (between 1 and 6) on different pieces of equipment in their department. For example, if I'm proficient at repairing a particular piece of equipment I will score ...
1
vote
0answers
21 views

Should categorical data be normalised in linear regression?

I have data similar to the following: [ [0, 4, 15] [0, 3, 7] [1, 5, 9] [2, 4, 15] ] I used One Hot Encoder to preprocess this data so it is suitable ...
0
votes
1answer
64 views

data normalization after dimension reduction for classification

The classifier is KNN or RBF-SVM. After doing dimension reduction (e.g., PCA, LDA or KPCA, KLDA), does it need to do normalization before classification? In LIBSVM ...
0
votes
0answers
79 views

normalization to zero mean and variance one logistic regression & random forrests

i was just thinking how does normalization to 0 mean and variance 1 (using an affine linear mapping) can impact the classification accuracy and the choice of hyperparameters when using: logistic ...
0
votes
1answer
48 views

Is it necessary to normalize data for hierarchical clustering of mixed variables using complete linkage?

I have a dataset with 3 numerical variables and 1 categorical variable which is binary (0,1). For clustering these data, should I normalize my numerical variables to the unit range (0,1) by ...
1
vote
1answer
44 views

Exponential family: examples where scaling constant is data dependent

The general form of a exponential family distribution is given as $$p(x|\theta) = h(x) g(\theta) \exp(\theta^Tu(x))$$ where $h(x)$ is referred to as the "scaling constant" (e.g. in Murphy's ML ...
0
votes
0answers
17 views

Error measure in regression task by using a neural network

I am working on a regression task in which I which I want to predict vectors of about 30 values starting from textual documents using a Convolutional Neural Network. In particular, for each document ...
0
votes
0answers
34 views

Normalise heart rate data

I need to statistically examine time series of heart rate data over many different users. Since I don't have data for each user for each instance, I should find a way to normalise the data so that I ...