A way of re-expressing data to make their values lie between 0 and 1 (or 0% and 100%).

learn more… | top users | synonyms (1)

0
votes
2answers
18 views

Normalization of dummy variables

My data consists of several continuous measurements and some dummy variables representing the years the measurements have been made. Now, I want to learn a neural network with the data. Therefore, I ...
1
vote
1answer
15 views

how to transform data of two experimental groups? one is positively skewed and one is negatively..

I have two experimental groups. Then I test their normality respectively. Result shows that one is positively skewed and the other is negatively skewed. In this case, how should I do the data ...
0
votes
0answers
15 views

Normalizing depth data

I have some Kinect data of somebody standing (reasonably) still and performing sets of punches. I am given it in the format of an x,y,z co-ordinate for each joint of which they are 20, so I have 60 ...
1
vote
2answers
22 views

How to transform negative data to be homoscedastic

I have a bunch of data that's both positive and negative. Its calculated from the residuals of an ANOVA (i.e. specific leaf area calculated as the residuals of an ANOVA of leaf area with leaf blade ...
0
votes
1answer
11 views

Normalization for pattern classification?

I'm working off my first independent project for some pattern classification. I'm utilizing some datasets from UCI machine learning, but am not sure on how to start with data normalization. The data ...
1
vote
1answer
28 views

SVM data normalization… what about classifying new (training) data?

I've got a big doubt about SVM classification task (and more in general classification task), about data normalization. Let's suppose I've a SVM trained with normalized data, and new data to classify. ...
1
vote
0answers
13 views

Pre-stimulus baseline removal in R

I have the following scenario: trials were conducted where participants were exposed to multiple stimuli during the course of a trial a specific physiological response was continuously recorded ...
0
votes
0answers
26 views

Scaling/Normalisation or Standardization

I'm working on SVM and ANN classification tools. In order to improve the classification accuracy, I want to know the best or the recommended data-preprocessing, is it scaling/normalisation or ...
1
vote
0answers
19 views

Normalization on frequently updated dataset

There are some normalization types like rescaling, standart score or modified standart score. I can apply these algorithms to large dataset. If the dataset that i am working on getting frequently ...
1
vote
0answers
23 views

The number of coin tosses needed if the proportion of heads is to lie within 0.05 of p with probability at least 0.9?

There's a question I'm not really sure if I did it right or even understand what its trying to say. There is a coin which produces heads with an unknown probability $p$. How many times should we ...
0
votes
0answers
18 views

corpus linguistics statistics

I am in doubt as to how calculate observed relative frequences in corpus linguistics. That's how I did it: I multiplied the number of words by thousand and then I divided it all by the size of the ...
1
vote
1answer
38 views

Comparison of subpopulations: do I need normalisation?

I have a population of people. Each person has one of three characteristics, say X, Y or Z. I want to compare other characteristics of these people, using the characteristics X, Y and Z as a ...
1
vote
0answers
38 views

Obtain the covariance matrix with the correlations and the variances

i have the following problem that makes me crazy. COnsider a trivariate normal variable, let $\sigma_1^2$, $\sigma_2^2$ and $\sigma_3^2$ be the variance of the three components and $\rho_{1,2}$, ...
2
votes
0answers
24 views

multiple linear regression with normalization - how to get non-scaled full covariance matrix

I am doing a quite complicated multiple regression modelling in physics and have a problem how to got back to covariance matrix for non-normalized parameters. I don't know how to calculate the error ...
0
votes
1answer
25 views

Normalize row or column while each row is an observation

Suppose I have a matrix compose of row as each observation, column as each property and I want to calculate the distance between each observation. In this case I think I should normalize each column, ...
0
votes
1answer
32 views

Normalize histogram with different bin width?

I know how to normalize histogram (so that the area =1) with the same bin width, but how to do it when the histogram has different bin width? Any idea?
1
vote
1answer
18 views

normalizing dataset for extrapolation - sample or population mean and standard deviation?

I am currently fitting models that are intended to be used for extrapolating from a limited sample to a large population. For a specific example, one model is predicting water temperature in rivers ...
0
votes
1answer
52 views

How do i normalize residuals?

I am trying to adjust a hierarchical multiple regression model and no matter which transformations I use (z-transformation, sqrt, cuberoot, inv, inv sqrt ...), I do not manage to get the residuals ...
0
votes
0answers
29 views

How to normalize ranked data

I am doing some machine learning and need help with the stats aspect of my problem. I have a number of addresses of webpages and some features for these webpages. I am running TF-IDF on the webpage ...
-1
votes
1answer
40 views

rescale a vector x to lie between arguments LOWER and UPPER in R [duplicate]

I am trying to rescales x to lie between lower and upper ...
0
votes
1answer
21 views

When training a RBM how should the data be normalised?

A feed forward neural network trains best if the data is normalised so that each input has a mean of 0 and a standard deviation of 1. Is this true for a Restricted Boltzmann machine as well? (My ...
0
votes
1answer
41 views

Time series degree of slope: calculating what I see

I want to calculate the degree of slope at each point in a time series. Different time series have different scales. The final number should be normalized in the range of +/-90 degrees. Basically, ...
0
votes
0answers
46 views

Calculation of normalization constant

This is an equation from the paper "A Content-based Probabilistic Correction Model for OCR Document Retrieval" - Rong Jin, Alex G. Hauptmann , ChengXiang Zhai $$P(w|M_{\text{orig}})= ...
1
vote
0answers
51 views

Unity normalization with respect to bounded variables

I have some difficulties to normalize a number of variables, such that their normalized equivalents respect some boundaries constraints. Suppose I have two variables $-1\leq w_1\leq 1$ and $-1\leq ...
0
votes
0answers
13 views

impact coding legitimity and rescaling

I have implemented a version of "impact coding" : http://www.win-vector.com/blog/2012/07/modeling-trick-impact-coding-of-categorical-variables-with-many-levels/. I have not see that name anywhere else ...
2
votes
1answer
48 views

Normalising higher moments of features for machine learning?

It's quite common to normalise different feature vectors so that they have the same mean and variance (usually (X-mean(X))/sd(X)), so that the changes in the ...
0
votes
0answers
72 views

Fitting normalized data to discrete distribution

I have a graph which represents a P(x) vs. x data where x can have 10 possible discrete outcomes. Data looks as follows: {{200, 0.0058668}, {306, 0.0503333}, {411, 0.163055}, {520, 0.203411}, {624, ...
0
votes
0answers
17 views

When do I need to normalize the data? [duplicate]

When do I need to normalize the data? What's the problem if I always normalize the data? For example, I am given a set of multivariate gaussian data. I use max. likelihood to estimate the estimate ...
4
votes
1answer
117 views

Is standardisation before Lasso really necessary?

I have read three main reasons for standardising variables before something such as Lasso regression: 1) Interpretability of coefficients. 2) Ability to rank the ...
0
votes
0answers
29 views

Problem with classifier prediction results

I built a classifier with 13 features ( no binary ones ) and normalized individually for each sample using scikit tool ( Normalizer().transform). When I make predictions it predicts all training sets ...
3
votes
0answers
19 views

Distribution of a vector divided by its sum [duplicate]

Suppose that the vector $\tilde{a} = (a_1,a_2,...,a_J)$ has a multivariate normal distribution and let $$A = \Sigma_{j=1}^J a_j\,.$$ Then what is the distribution of $a_j/A$ or ...
1
vote
2answers
93 views

How to improve a Fraud Classification Model?

I built a classification model (Logistic Regression) in order to classify data in Fraud or Not Fraud. This data is related with online CNP (Card Not Present) transactions and after choosing some ...
2
votes
0answers
26 views

Is it okay to take the average of several normalized time series?

I have 3 time series (10 years) of economic data for 3 countries. For another analysis that I am doing, I should try and reduce these into preferably 1 time series. For this analysis I don't care so ...
1
vote
0answers
22 views

range normalized with unclear highest value

I tried to normalize many set, which each set consists of 12 values. I want to normalize it into range between 1 and -1. However, the highest value is unclear because the value can be trillion, or ...
4
votes
2answers
76 views

Should normalization completely weed out correlation?

I have two variables: ordering & length. The former measures the ordering of a sequence (i.e. all permutations of A-B-C), and the former is the length of the sequence (i.e. A-B-C has a length of ...
1
vote
0answers
36 views

Normalization of circularly-symmetric complex Gaussian distribution

I have a hard time describing my problem, but I'll try my best. It's all about the well-known zero-mean, circularly-symmetric, multivariate complex Gaussian distribution ...
0
votes
1answer
73 views

data normalisation problems

I am pretty new in machine learning and hence facing a lot of confusion in data normalisation concepts. Someone pls clarify the following doubts : 1) While normalising a data matrix of m-samples x ...
2
votes
2answers
64 views

Is normalizing the features always good for classification?

I always read in books that when we do classification or machine learning tasks it's always better to normalize the features so to make them in one range like 0-1. Today I used weka to play with Iris ...
4
votes
2answers
123 views

How to normalize Poisson distributed data before PCA

I am a newbie to principal component analysis (PCA). I will have to do PCA to data sets consisting of count statistics: all data are positive integers. Before PCA the data needs to be normalized. It ...
0
votes
1answer
39 views

Normalization of attributes in test set for a neural network

I have built a neural network classification model and each attribute is z-normalized when building the model by the following X=(x-training set sample mean)/training set sample deviation. If I now ...
0
votes
1answer
59 views

What are the pitfalls of using subjective ratings for correlation and causal explanation?

Take a study that collects subjective guesstimates as a proxy for some variable, like the quality of service. In a simplest model, we then regress the monthly sales on these guesstimates. How to ...
2
votes
1answer
296 views

Normalizing to zero mean and unit variance before regression

I'm new to regression (vector autoregression), and recently encountered the following issue: If I use raw dependent and independent variables to do the regression, the $R^2$, DW-d test and standard ...
0
votes
0answers
17 views

Comparing sample distributions. What should I normalize by for lower population sample distributions?

I'm sorry if I'm not describing this accurately. I have a distribution of lengths of a protein. For proteins that are near the average length, I have many more data points, considering there are ...
1
vote
1answer
138 views

Normalizing difference between two real values to [0,1] interval [duplicate]

If I have two positive real numbers that can take on any value between 0 and some finite real number, how do I normalized the difference between these two numbers to [0,1] interval where 0 indicates ...
1
vote
0answers
60 views

Do I need to rescale dummy variables for PCA?

I am wondering if for a PCA for which I rescale my numeric variables, I need to rescale my dummy variables as well ? I have read on the internet that I should not but I do not see why. I guess the ...
0
votes
0answers
22 views

Normalization and hypothesis testing under system equations

Hypothesis testing under single equation linear regression is robust to simple data normalization (for example dividing all variables by their respective mean). I see that the same is true for systems ...
0
votes
0answers
68 views

Standard score (z-transformation) normalization or 0-1 normalization, which one is better for k-NN?

I am not much experienced in data mining, but I know that I should normalize my data before running k-NN classifier on it, to have reasonable results. But I found out, that there are many methods to ...
0
votes
0answers
43 views

Alignment and comparison of two unimodal and one uniformly distributed datasets

This question is similar to the following question: Normalize 3 irregulary distributed datasets and make their datapoints statistically relevant to each other describes similar problem, but is more ...
0
votes
0answers
76 views

Normalize 3 irregulary distributed datasets and make their datapoints statistically relevant to each other

A very practical case. I have 3 parallel inputs/aka data-sets, or you may also call it: A large set of learn cases, each one having 3 parameters/inputs (A, B and C) All 3 inputs have the same ...
2
votes
2answers
182 views

Perform feature normalization before or within model validation?

A common good practice in Machine Learning is to do feature normalization or data standardization of the predictor variables, that's it, center the data substracting the mean and normalize it dividing ...