A way of re-expressing data to make their values lie within a specified range.

learn more… | top users | synonyms (1)

2
votes
0answers
16 views

Scale a set of probabilities so their sum is a target value

I have a set of probabilities; e.g. $p = (0.95, 0.9, 0.6)$ I want to scale them up so that their sum moves from $\sum p = 2.45$ to $\sum p^* = 2.5$. I think I need normalise to ensure the rescaled ...
0
votes
2answers
14 views

Ranging [0,1] test set with parameters from training set

I am working on Machine Learning, particularly I have a dataset with 50+ columns and 100,000 rows. I need to get the data normalized with ranging to [0,1] (not with standardization) and I've split the ...
0
votes
0answers
16 views

Should data be normalized before or after imputation of missing data?

I am working on a metabolomics data set of 81 samples x 407 variables with ~17% missing data. I would like to compare a number of imputation methods to see which is best for my data. Is there a ...
2
votes
0answers
33 views

Intuition of Bayesian normalizing constant

In the commonly mentioned mammography screening problem with a screening likelihood of 80%, a prior of 10% and a false positive rate of 50%, or its variants, it is easy to explain that the conditional ...
0
votes
0answers
16 views

Effect of feature normalisation w.r.t. another feature in machine learning tasks (Regression, classification)

Let's say we have a set of features, and in this set of features there is one which is highly correlated to the others. What would be the implication of normalising the other features with respect to ...
0
votes
2answers
17 views

How to normalize feature vectors for concatenating

I have two different feature vectors of completely different scale, which are to be used as training data for machine learning algorithm. When I concatenate them, should I scale and normalize them ...
0
votes
0answers
19 views

Self Organizing Map and input normalizing

I've been playing around with self organizing maps (SOM) recently. I tried to implement a simple example. You can see the training implementation function gist here and full contained SOM example ...
0
votes
0answers
8 views

Is normalization required in Sammon mapping

I have a data set of 480 samples with 7-dimensions and I want to implement a Sammon mapping into 3-dimensions. In Principal Component Analysis to my understanding we need to normalize the data in ...
0
votes
2answers
38 views

Testing for Normality

As part of an assignment I have to do a leakage study for a chemical product. We have been provided with data from 8 different batches, with 12 observations from each batch. The observed variable ...
0
votes
0answers
8 views

How to normalize dprime values in a discrimination task?

In a 2AFC discrimination task, what would be an appropriate method for normalization of dprime values of different subjects when I want to show that certain conditions are different or identical given ...
1
vote
1answer
26 views

In a class with multiple teachers, how can I transform student scores based on their teacher's average compared to the population average?

It has been ages since I've taken any statistics courses, and I have found myself in the following situation: I am in charge of a university course with about 400 students and 10 assessors. There ...
0
votes
3answers
70 views

Normalize data input after training a neural network

I have a simple question. I am training a neural network feeding it with normalized data patterns using Gaussian normalization. My question arises when I see that some people use the mean and standard ...
0
votes
1answer
27 views

Support vector regression (LIBSVM) returns out of range outputs when I use out-of-sample data to predict one step ahead (MATLAB)?

I'm using SVR model in MATLAB R2016a using this option: options_z = ['-q -s 3 -t 2 -c ', C_param, ' -p ', epsilon, ' -g ,Kernel_scale]; I'm optimizing SVR ...
0
votes
0answers
6 views

convergence of coordinate descent applied to lasso

When using coordinate descent for solving a lasso regression, does normalizing the features impact the convergence rate?
1
vote
0answers
20 views

How to perform quantile transformation with missing values?

Given are an input vector $I$ with missing values and a target/reference distribution ${T}$. For example: $I$: 0.215 NA 0.103 0.649 0.057 0.292 NA 0.433 0.521 NA $T$: -0.996 -0.606 -0.394 -0.090 ...
1
vote
0answers
19 views

Is output data normalization necessary in SVM regression?

We talk a lot about input data normalization, I want to know if output data normalization can do good to SVM regression, for example, maybe it could help to reduce grid search scope when doing ...
0
votes
0answers
39 views

Transformation of data for normality

I'm intended to run a linear regression model (Rain~dBZ) for my data set. I would like to know how to transform non-normal set of "Rain" column in to a normal distribution. I would really appreciate ...
0
votes
0answers
49 views

understanding Batch Normalization

I'm trying to make sense of the Batch Normalization paper, in Section 3.2, it says We could have also normalized the layer inputs u, but since u is likely the output of another nonlinearity, the ...
1
vote
0answers
46 views

What is the efficient preprocessing data in image classification task with CNN?

I am new in deep learning on image classification. I know that Machine learning algorithm are very dependent to data normalization. Usually, if we have a training data set represented with X [N*D] ...
0
votes
0answers
19 views

Using trimmed means and Winsorized variances to compute standardisation of data

I am looking into the pros and cons of each normalisation technique for work and it got me thinking. What if I used trimmed means and the sqrt of Winsorized variances to compute the standardised data? ...
1
vote
0answers
15 views

Proxies and uses for the Geometric Mean of negative (or even complex) data

I use the geometric mean (GM) as a scale factor for data normalization. To avoid the $0$ cancellation effect with positive values, I use the simple offset GM: $$ \hat{x} = ...
0
votes
0answers
16 views

Rescaling vs Standardization of features

Is there any general rule of thumb or any justified rule to choose whether to scale a dataset using Rescaling (for each feature, subtract the min value and divid by the max - min) or Standardization ...
0
votes
1answer
22 views

Normalize all data before cross-validation or normalize every train part separately and use same properties for test part?

Suppose that we want use 5-fold cross-validation for a support vector regression(SVR) model. We should normalize total data before cross-validation process or we need normalize every train part ...
1
vote
1answer
30 views

Normalising features extracted using a CNN?

I have used a pre-trained CNN to extract features from training and test images sets. The same CNN was used for all images. The CNN includes normalization layers. Before training a classifier (SVM ...
1
vote
1answer
22 views

Normalizing the Turbulence index to match other indices

I calculated the index of complexity, longitudinal entropy, and turbulence for my data. The first two indicators vary from 0 to 1, whereas turbulence varies between 1 and 16 for one dataset and ...
0
votes
1answer
17 views

How to find the unit-normalised form of the distribution?

If we have P($x_1,x_2,x_3$) $\propto$ $\delta_D$($x_1+x_2+x_3$-1) $\Theta(x_1)$$\Theta(x_2)$$\Theta(x_3)$, then how to find the form of P($x_1,x_2,x_3$)? i.e., how to calculate the integral ...
0
votes
1answer
17 views

Best way to examine longitudinal data?

I had 20 patients come to clinic once a month for 6 months. At each visit we collected baseline data. We then gave the patients 3 different treatments to see the effects for each visit. Thus we have ...
1
vote
1answer
37 views

How Do I Rank the GPA of Students from Different Colleges?

I have data from 100 students, each at a different college would like to rank them by GPA. I have the following data for each: The student's GPA: example 3.25 The all student average: 3.10 The ...
1
vote
1answer
94 views

Normalization of count data of time periods with different length

I have count-data from two time-periods which differ in length. The event I'm counting is in both periods the same kind of event. Period 1 is 120 hours Period 2 is 48 hours At the end I have ...
0
votes
1answer
22 views

MinMax normalization when all elements are same

I'm using min-max normalization to normalize time series which I compare in the following. My question is, by definition min-max normalization is defined as: ...
0
votes
1answer
15 views

Correlation of levels vs. differences vs. percents

Sometimes, I have seen people using correlation of levels, correlation of differences and also correlation of percent changes. I understand these answer different questions. For example, for "what is ...
0
votes
0answers
12 views

Scaling existing regression coefficients to predict for different dataset

I am tasked with coming up with a way to project customer activity for different groups of customers for 60 months. These groups can be based on a multitude of factors - plans, acquisition channel, ...
1
vote
0answers
27 views

How to deal with correlated response variables

I have five response variables measured at $2$ different times and I would like to know on which variables there is an improvement. I could simply divided the newest responses on the old ones and ...
0
votes
1answer
32 views

Normalization vs Standardization for multivariate time-series

I'm using DTW as a distance measure for comparing two multivariate time-series. I want to be able to cluster data using DTW as distance measure, since time-series may be shifted, skewed. Since there ...
2
votes
0answers
108 views

double feature value in ridge regression, coefficients change?

In ridge regression using unnormalized features, if you double the value of a given feature A (i.e., a specific column of the feature matrix), what happens to the estimated coefficients for every ...
0
votes
0answers
24 views

Normalize data by features or by instance?

Before applying Kernel tricks in SVM, I know I should normalize data at first. (1) But should I normalize data by features or by instances? For example, if I have n data points, each data point has d ...
2
votes
1answer
109 views

How to Normalize data?

In a study, the baseline data are recorded (blood pressure, BP1) prior to the experiment (watching a horror film). After the experiment, the data (BP2) are collected again. The problem is that not all ...
0
votes
0answers
30 views

Neural network can't learn to predict small changes

I design neural network for a prediction of numbers. I want to predict trend of a visit rate of articles. This number has big range [0,7000]. I scale numbers with min-max normalization to range ...
1
vote
0answers
53 views

Which is better to normalize data? [duplicate]

I saw on Coursera machine learning classes that is possible to normalize data in two ways: data = (data - mean) / max(data) - min(data) or you can use an Octave ...
1
vote
0answers
19 views

Normalization by z-score + range [0,1]

I am trying to normalize my dataset for further analysis. I have several data coming from different subjects, so I have first applied z-score normalization to each variable of each subject in order to ...
1
vote
1answer
25 views

Do I apply normalization per entire dataset, per input vector or per feature?

One of the ways to standardize input data for Neural network training is: \begin{equation} X = \frac{X - mean(X)}{std(X)} \end{equation} However if I have have $n$ training examples which have each ...
4
votes
2answers
105 views

When to normalize data in regression? [duplicate]

Under what circumstances should the data be normalized/standardized when building a regression model. When i asked this question to a stats major, he gave me an ambiguous answer "depends on the data". ...
0
votes
0answers
7 views

compare data from different sampling conditions

I did the same experiment on two different days. On each day I have two groups - the test and the control group. Due to changes in conditions (temperature etc.), the measurements each day came up on ...
0
votes
2answers
66 views

Use a combination of grand mean and group mean centering to standardize variables

I'm using cluster analysis to examine profiles of three variables, X1, X2, and X3. Because ...
0
votes
1answer
46 views

Normalization of values

How do I normalize a set of data? Let's say we are looking at measurements of diameter of the heart in women and men. Men will have bigger hearts, so error in measuring by the observers will always be ...
0
votes
1answer
41 views

In general, does normalization mean to normalize the samples or features?

I'm just getting into machine learning, and I have seen two conflicting practices for normalization. To be concrete, let's suppose that we have a $n \times d$ matrix containing our training data, ...
0
votes
0answers
26 views

Normalization of data between -1 and 1 [duplicate]

I would like the values -2,-1,1,2,3,4 to be normalized in the range -1 to 1. Can someone please help me with the formula to do this?
0
votes
0answers
23 views

Predicting class on data with a distribution that is different than that on which the classifier was trained

I have trained a random forest classifier on quantile-normalized data (gene expression: 20000 variables, 200 samples, RMA preprocessing). Goal: Using this classifier, I want to predict the class of ...
0
votes
1answer
27 views

Row normalization before correlation analysis for abundance data

I work with datasets in which protein abundances are reported across samples. I have some measurements of biological samples that should be more or less equal in protein abundance. After getting ...
0
votes
1answer
40 views

How to transform an arcsine distribution to a normal distribution?

I have a distribution that looks like this: U In other words small and big values are more frequent than middle values. A better graphical example of the distribution is this: Here So I have a ...