A way of re-expressing data to make their values lie within a specified range.

learn more… | top users | synonyms (1)

0
votes
0answers
11 views

Is centroid came from normalized or reduced data still valid?

I want to incorporate clustering to simplify data and speed up classification execution time. Let's say I have data like this : ...
1
vote
1answer
33 views

normalisation in k means clustering on percentages and other numerical variables

I have several variables to include in k-means, some of them are percentages (between 0-1) and some of them are numerical variables (positive values). I know normalisation is required when the ...
3
votes
1answer
44 views

Normalise X/Y coordinates to stop jitter

Disclaimer: I am a programmer by trade, not a statistician, so please cater to my ignorance and I apologize now if I make any incorrect assumptions Please consider the following problem: I am using ...
1
vote
0answers
27 views

Regression analysis method when data is linked to a normalised time… and only hold a relationship for some of it

Stats novice... Please be patient, but I would really appreciate some help. I am using SPSS. I have made some kinematic measurements of a closed chain, repeated movement. I have a large number of ...
0
votes
0answers
17 views

How to Normalise data on a Likert scale 1-7

I'm looking to create environmental and social sustainability values for countries, similar to what is carried out here. http://reports.weforum.org/global-competitiveness-report-2014-2015/appendix-a/#...
0
votes
0answers
16 views

z-score normalisation - how to achieve this for score combination?

I have read that to perform a score fusion from two different classifiers on two different datasets then the score must be normalised. I understand I can use z-score normalisation / max-min ...
2
votes
1answer
36 views

Data normalization or not

I have one data set with several variables and some of them have wide range. I want to predict sales from historical data. Sales from 0 to 6000000, epoch time like 1419656400 in seconds but I ...
0
votes
0answers
59 views

Data normalization/standarization and comparing

I have a question related to data comparing. First of all, my dataset is composed by cities of the world. In this cities, we have a maximum of 24 tags that indicate what these cities are best for. ...
1
vote
0answers
10 views

Should word embedding vectors be normalized before being used as inputs? [duplicate]

To keep all the inputs to a network on the same scale, they are usually normalized so that they end up being represented as number of standard deviations from the mean. Is this something that needs ...
0
votes
0answers
12 views

normalization and rescaling definitions are mixed up?

in: https://en.wikipedia.org/wiki/Normalization_(statistics) it's written that rescaling(min_max_scaling) and standarization are types of Normalization what i see in alot of stackoverflow answers is ...
1
vote
0answers
12 views

How to compare number of incidents in different geographical areas when they have different population size?

I'm trying to make comparisons between number of incidents that happened in different zip codes. The goal is to figure out whether those incidents happen more frequently in certain areas than others. ...
2
votes
1answer
32 views

Power Calculations using “Percent Change” vs. “Absolute Values”

Percent change can be extremely misleading for a number of reasons (e.g. regression to the mean), even though in the medical literature percent change is used quite frequently to "normalize" clinical ...
0
votes
0answers
26 views

Normalizing year data

I have a data set in which there are data from April 2010 to Dec 2010 ( 9 Months) Jan 2011 to Dec 2011 (12 Months ) Jan 2012 to Dec 2012 (12 Months) Jan 2013 to Dec 2013 (12 Months) Jan 2014 to ...
1
vote
2answers
42 views

Transfer entropy value between 0 and 1

Given two variables, X and Y, there is a way of obtaining a Mutual Information value between 0 and 1 by: MI_normalised=MI_original/sqrt(H(X)*H(Y)); where H(X) ...
6
votes
3answers
144 views

Raw data outperforms Z-score transformed data in SVM classification

I've been trying to perform a binary classification using an SVM classifier (scikit-learn's SVC with RBF kernel). I have a sample size of about 100, with about 70 features each. The features are of ...
0
votes
0answers
12 views

picking a kernel SVM, how to normalize categorical + numerical data

So I have a dataset that contains both categorical and numerical data for each data point, and a class for each data point. My goal is to plan to build an SVM model from the data to predict the class ...
3
votes
0answers
39 views

Scale a set of probabilities so their sum is a target value

I have a set of probabilities; e.g. $p = (0.95, 0.9, 0.6)$ I want to scale them up so that their sum moves from $\sum p = 2.45$ to $\sum p^* = 2.5$. I think I need normalise to ensure the rescaled ...
0
votes
2answers
21 views

Ranging [0,1] test set with parameters from training set

I am working on Machine Learning, particularly I have a dataset with 50+ columns and 100,000 rows. I need to get the data normalized with ranging to [0,1] (not with standardization) and I've split the ...
0
votes
0answers
30 views

Should data be normalized before or after imputation of missing data?

I am working on a metabolomics data set of 81 samples x 407 variables with ~17% missing data. I would like to compare a number of imputation methods to see which is best for my data. Is there a ...
4
votes
2answers
164 views

Intuition of Bayesian normalizing constant

In the commonly mentioned mammography screening problem with a screening likelihood of 80%, a prior of 10% and a false positive rate of 50%, or its variants, it is easy to explain that the conditional ...
0
votes
0answers
17 views

Effect of feature normalisation w.r.t. another feature in machine learning tasks (Regression, classification)

Let's say we have a set of features, and in this set of features there is one which is highly correlated to the others. What would be the implication of normalising the other features with respect to ...
0
votes
2answers
38 views

How to normalize feature vectors for concatenating

I have two different feature vectors of completely different scale, which are to be used as training data for machine learning algorithm. When I concatenate them, should I scale and normalize them ...
0
votes
0answers
24 views

Self Organizing Map and input normalizing

I've been playing around with self organizing maps (SOM) recently. I tried to implement a simple example. You can see the training implementation function gist here and full contained SOM example ...
0
votes
0answers
9 views

Is normalization required in Sammon mapping

I have a data set of 480 samples with 7-dimensions and I want to implement a Sammon mapping into 3-dimensions. In Principal Component Analysis to my understanding we need to normalize the data in ...
0
votes
2answers
43 views

Testing for Normality

As part of an assignment I have to do a leakage study for a chemical product. We have been provided with data from 8 different batches, with 12 observations from each batch. The observed variable ...
0
votes
0answers
9 views

How to normalize dprime values in a discrimination task?

In a 2AFC discrimination task, what would be an appropriate method for normalization of dprime values of different subjects when I want to show that certain conditions are different or identical given ...
1
vote
1answer
28 views

In a class with multiple teachers, how can I transform student scores based on their teacher's average compared to the population average?

It has been ages since I've taken any statistics courses, and I have found myself in the following situation: I am in charge of a university course with about 400 students and 10 assessors. There ...
0
votes
3answers
88 views

Normalize data input after training a neural network

I have a simple question. I am training a neural network feeding it with normalized data patterns using Gaussian normalization. My question arises when I see that some people use the mean and standard ...
0
votes
1answer
33 views

Support vector regression (LIBSVM) returns out of range outputs when I use out-of-sample data to predict one step ahead (MATLAB)?

I'm using SVR model in MATLAB R2016a using this option: options_z = ['-q -s 3 -t 2 -c ', C_param, ' -p ', epsilon, ' -g ,Kernel_scale]; I'm optimizing SVR ...
0
votes
0answers
14 views

convergence of coordinate descent applied to lasso

When using coordinate descent for solving a lasso regression, does normalizing the features impact the convergence rate?
1
vote
0answers
25 views

How to perform quantile transformation with missing values?

Given are an input vector $I$ with missing values and a target/reference distribution ${T}$. For example: $I$: 0.215 NA 0.103 0.649 0.057 0.292 NA 0.433 0.521 NA $T$: -0.996 -0.606 -0.394 -0.090 -0....
1
vote
0answers
21 views

Is output data normalization necessary in SVM regression?

We talk a lot about input data normalization, I want to know if output data normalization can do good to SVM regression, for example, maybe it could help to reduce grid search scope when doing cross-...
0
votes
0answers
39 views

Transformation of data for normality

I'm intended to run a linear regression model (Rain~dBZ) for my data set. I would like to know how to transform non-normal set of "Rain" column in to a normal distribution. I would really appreciate ...
1
vote
0answers
77 views

Why do we batch normalize the pre-synaptic values rather than the activations when using batch normalization?

I'm trying to make sense of this batch normalization (1) paper, in Section 3.2, it says We could have also normalized the layer inputs u, but since u is likely the output of another nonlinearity,...
1
vote
0answers
62 views

What is the efficient preprocessing data in image classification task with CNN?

I am new in deep learning on image classification. I know that Machine learning algorithm are very dependent to data normalization. Usually, if we have a training data set represented with X [N*D] ...
0
votes
0answers
24 views

Using trimmed means and Winsorized variances to compute standardisation of data

I am looking into the pros and cons of each normalisation technique for work and it got me thinking. What if I used trimmed means and the sqrt of Winsorized variances to compute the standardised data? ...
1
vote
0answers
16 views

Proxies and uses for the Geometric Mean of negative (or even complex) data

I use the geometric mean (GM) as a scale factor for data normalization. To avoid the $0$ cancellation effect with positive values, I use the simple offset GM: $$ \hat{x} = \sqrt[n]{\prod\limits_{i=1}^...
0
votes
0answers
17 views

Rescaling vs Standardization of features

Is there any general rule of thumb or any justified rule to choose whether to scale a dataset using Rescaling (for each feature, subtract the min value and divid by the max - min) or Standardization (...
0
votes
1answer
28 views

Normalize all data before cross-validation or normalize every train part separately and use same properties for test part?

Suppose that we want use 5-fold cross-validation for a support vector regression(SVR) model. We should normalize total data before cross-validation process or we need normalize every train part ...
1
vote
1answer
34 views

Normalising features extracted using a CNN?

I have used a pre-trained CNN to extract features from training and test images sets. The same CNN was used for all images. The CNN includes normalization layers. Before training a classifier (SVM ...
1
vote
1answer
23 views

Normalizing the Turbulence index to match other indices

I calculated the index of complexity, longitudinal entropy, and turbulence for my data. The first two indicators vary from 0 to 1, whereas turbulence varies between 1 and 16 for one dataset and ...
0
votes
1answer
18 views

How to find the unit-normalised form of the distribution?

If we have P($x_1,x_2,x_3$) $\propto$ $\delta_D$($x_1+x_2+x_3$-1) $\Theta(x_1)$$\Theta(x_2)$$\Theta(x_3)$, then how to find the form of P($x_1,x_2,x_3$)? i.e., how to calculate the integral $\...
0
votes
1answer
18 views

Best way to examine longitudinal data?

I had 20 patients come to clinic once a month for 6 months. At each visit we collected baseline data. We then gave the patients 3 different treatments to see the effects for each visit. Thus we have ...
1
vote
1answer
41 views

How Do I Rank the GPA of Students from Different Colleges?

I have data from 100 students, each at a different college would like to rank them by GPA. I have the following data for each: The student's GPA: example 3.25 The all student average: 3.10 The ...
1
vote
1answer
111 views

Normalization of count data of time periods with different length

I have count-data from two time-periods which differ in length. The event I'm counting is in both periods the same kind of event. Period 1 is 120 hours Period 2 is 48 hours At the end I have ...
0
votes
1answer
24 views

MinMax normalization when all elements are same

I'm using min-max normalization to normalize time series which I compare in the following. My question is, by definition min-max normalization is defined as: ...
0
votes
1answer
21 views

Correlation of levels vs. differences vs. percents

Sometimes, I have seen people using correlation of levels, correlation of differences and also correlation of percent changes. I understand these answer different questions. For example, for "what is ...
0
votes
0answers
13 views

Scaling existing regression coefficients to predict for different dataset

I am tasked with coming up with a way to project customer activity for different groups of customers for 60 months. These groups can be based on a multitude of factors - plans, acquisition channel, ...
1
vote
0answers
28 views

How to deal with correlated response variables

I have five response variables measured at $2$ different times and I would like to know on which variables there is an improvement. I could simply divided the newest responses on the old ones and ...
0
votes
1answer
39 views

Normalization vs Standardization for multivariate time-series

I'm using DTW as a distance measure for comparing two multivariate time-series. I want to be able to cluster data using DTW as distance measure, since time-series may be shifted, skewed. Since there ...