A way of re-expressing data to make their values lie between 0 and 1 (or 0% and 100%).
2
votes
2answers
58 views
Why Normalization (Standardization) values should be smaller than $1$?
The books gives some examples about content based recommendation. An example of what I understood is at below.
A movie's attributes are values between $1$ and $10$. The duration attribute gets ...
4
votes
3answers
103 views
Should you ever standardise binary variables?
I have a data set with a set of features. Some of them are binary (1=active or fired, 0= inactive or dormant) and the rest are real valued, i.e. 4564.342.
I want to feed this data to a machine ...
0
votes
0answers
67 views
Transform non-normal data to normality by rescoring columns
I have a vector with fluency of translation - (0, 0, 1, 3, 3, 3 ,3 ....)
The problem is that it is made by people (for example someone gives too much 3 but only a bit of 2) and we want to normalize ...
1
vote
1answer
29 views
Normalization of three different variables for linear addition
I am a research student and stuck at a point in my work and want your help. I have three different variables for each node of my network (Energy, Traffic Load and Link quality). These three variables ...
0
votes
1answer
29 views
Data normalization before giving to Neural Nets or Deep Learning algorithm?
What kind of normalization scheman is required for the best of NN algorithms? I saw some people just give the data to signum function before passing to NN and some of those process data by regular ...
1
vote
1answer
35 views
Reshaping a distribution
Not sure what the exact term is for what I'm trying to do.
I have a data set with random variable x with values X1, X2, ..., XN that has a standard deviation sigma and a mean m.
I want to perturb ...
1
vote
1answer
36 views
How to perform Normalization on Call Details Record to perform k-Mean Clustering
I'm new to data mining and currently doing mining project on telecom customer segmentation (based on profile and call details record). I have gender, age, call time and call duration and have to ...
2
votes
1answer
51 views
Confusion related to data normalization
I am trying to learn a linear regression model. However, I have some confusion related to the normalization of the data. I have normalized the features/predictors to zero mean and unit variance. Do I ...
1
vote
1answer
51 views
Normalization Factor Wrong? (Bug?)
I'm new to PyMC and Bayesian stuff in general, so I started off with what I thought was a very simple toy problem. I generated some normally-distributed noise with a given mean and standard ...
1
vote
0answers
21 views
getting rid of the batch effects
I have a two data set one with n=15 another with n=25.Each set of data has around 90 variables.The data collected was experimental so when i try to combine my two datasets i see clear differences in ...
0
votes
1answer
27 views
Normalization / Moving Average
I have this daily time series of observed prices: $P_1,P_2,..., P_n$.
I want to works with returns: $ 0 , P_2-P_1,..., P_n - P_{n-1}$.
I have been told to "remove" the first term (P_1-P_0= P_1- ?) ...
0
votes
1answer
52 views
Is there a way in python/java/scala to convert/normalize log normal distribution into normal distribution?
We have a data set that looks like a lognormal distribution when we plot it.
We would like to convert/normalize the distribution into normal distribution and see what feature weight got enhanced.
It ...
0
votes
1answer
102 views
Dynamic Time Warping and normalization
I'm using Dynamic Time Warping to match a "query" and a "template" curve and having reasonable success thus far, but I have some basic questions:
I'm assessing a "match" by assessing whether the DTW ...
0
votes
0answers
52 views
Normalizing SVM predications to [0,1]
I have trained an linear SVM which takes a pair of objects, computes features and is expected to learn a semantic similarity function between objects(we can say that it predicts whether the two ...
1
vote
2answers
164 views
How to deal with extreme but “real” data, classify as outliers or no?
I have an explanatory variable, close, which is the daily close price of a firm in the stock market.
The following summarizes this explanatory variable:
...
2
votes
1answer
27 views
When to normalize learning?
I'm trying to determine the effect of three types of learning on a group of subjects.
I have their pretest scores and posttest scores.
The current goal is to determine which intervention reduce the ...
2
votes
1answer
106 views
LOESS and MA normalization in R?
Attempting to do loess on two variables x and y in R using MA normalization (http://en.wikipedia.org/wiki/MA_plot) like this:
...
1
vote
0answers
73 views
Test to compare means normalized to different controls
I am running experiments on cells which are given two different treatments. Treatment one is a control virus, and treatment two is an active virus. Due to the variability between cell lines, I always ...
0
votes
1answer
133 views
Converting spectral data to RGB and normalizing appropriately
I have spectral data which tells me how strong of a response in RGB values for some color-space I get for a particular wavelength of light.
Typically, if I want "plain" white light, I'd have to sum ...
5
votes
1answer
126 views
How can I devise a scoring system for a competition that is more fair than straight percentages?
I am trying to come up with a method for deciding the winner from among eight student groups competing for a prize.
The raw data and corresponding percentages measure participation per group in a ...
0
votes
2answers
117 views
Data normalization and classification
I hope this clearly states the problem I have in hand. Here goes:
I've trained a neural network with one initial data set that was normalized in order to guarantee an equal participation of each ...
2
votes
2answers
160 views
how to avoid 0 determinant when sample covariance matrix has very small values
I have $n$, $p$-dimensional vectors and I am construction the $p$x$p$ covariance matrix using the following formula:
$Cov(j,k) = {1/(n-1)} {{\sum^n_{i=1}} (x_i(j) - {\mu}(j)) * (x_i(k) - {\mu(k)}) ...
1
vote
2answers
212 views
It it legitimate to apply a one-way ANOVA to data that have been normalised to the untreated controls?
I was wondering if it is OK to use one-way ANOVA after normalisation to the untreated controls? This is in an animal model of wound healing where there are 4 wounds per animal, one untreated and 3 ...
0
votes
0answers
26 views
Garage Benchmark
I'm using my garage as an experiment in learning how to write a simple benchmarking software but I'm not the best at statistics. Maybe someone can help me here.
My garage is refrigerated 100% of the ...
2
votes
1answer
105 views
How to calculate expected win percentage where only 2 of 5 values are known?
The idea here is that you have a 5 vs. 5 game where each player is using a unique character (henceforth 'hero'), and thousands of matches of this game have been played. The goal of the analysis is to ...
0
votes
0answers
40 views
Normally and exponentially distributed features
I have several features of which some are exponentially distributed and some are normally distributed.
Can I / should I used both at the same for training a classifier (e.g. neural net) and how ...
0
votes
0answers
114 views
Question about normalizing time series data samples
I have a set of times series biometric data samples for a number of different subjects. In each data sample subjects perform a specified action, but the time spent performing the action differs based ...
1
vote
1answer
194 views
Can I use z-scores to compare results from different measures?
Can I compare outcome data from different literacy measures with different scales (delivered to different subjects) by transforming the data into z scores? Ideally I'd like to use regression and ...
1
vote
1answer
71 views
Z scores derived from a regression equation in one group applied to other groups
I work in the developmental psychology arena.
Often papers report using regression to derive a function of age onto a score in a typical group and then applying this function to a second atypical ...
1
vote
1answer
57 views
Limit of quantile normalization
Is quantile normalization adequate for normalizing
data with very few samples?
For example this microarray data. Typically after normalization
we'd like to compare ...
1
vote
2answers
175 views
Confused about proper way to normalize two variables
I have two variables of interest:
Residential Vacancies (res_vac)
Commercial Vacancies (com_vac)
I also have two variables with which I might normalize the above:
Total Residences (res_tot)
...
1
vote
1answer
126 views
Whitening and unwhitening for sparse coding
Is this procedure for whitening and unwhitening correct?
Given an image $i$:
decompose the image in patches:
patch=im2col(i,[8 8],'sliding');
Whitening step:
...
0
votes
0answers
34 views
Does the standard scores analysis make sense in the following scenario?
INTRODUCTION
I've got an image in grayscale.
Say the image is a rectangle with width = 1700 pixels and height = 2338 pixels.
The image is light in the middle and dark in the top and bottom edges.
The ...
3
votes
2answers
232 views
does it make sense for non-negative data to subtract the mean and divide by the std dev?
It is a very usual procedure to subtract the mean and divide by the standard deviation in a set of data. If we deal with non-negative data, i.e. image, (in [0,1] or [0,255]), does this procedure make ...
3
votes
2answers
176 views
What are good initial weights in a neural network?
I have just heard, that it's a good idea to choose initial weights of a neural network from the range $(\frac{-1}{\sqrt d} , \frac{1}{\sqrt d})$, where $d$ is the number of inputs to a given neuron. ...
1
vote
1answer
170 views
Process for Standardising and Normalising data
First: I'm not well versed in statistics terminology so please forgive me - I'll try to be as verbose as possible with my problem.
This is a problem which I've previously solved very naively. I'm ...
-3
votes
1answer
167 views
On what basis do we differentiate a standard normal distribution from normal distribution?
The normal distribution is a prerequisite for ANOVA. It is not clear what we are supposed to understand by "standard normal distribution". Does ANOVA need a standard normal distribution? The answer ...
0
votes
0answers
61 views
Should I normalize for internal standard by taking residuals or by including the standard in the model?
I have some mass-spectroscopy data from several dozen samples, with the abundance of 60 compounds reported for each sample. Four internal standards were run with each sample, and their abundances are ...
1
vote
0answers
67 views
How do you predict the value of new instance, when the training data were normalized?
I estimated a Partial Least Squares model where the X matrix had normalized columns. Now I want to predict the value for a new instance (which is a frequency vector summing to one.) I assume that if I ...
0
votes
1answer
92 views
Is it needed to normalize data before rule model extraction algorithms like ID3?
I will use naive Bayes or decision tree that gives rule model both. Is is necessary to normalize data before working with such algorithms.
1
vote
0answers
50 views
What is the best mathematical transformation for a variable with many zero values? [duplicate]
Possible Duplicate:
How should I transform non-negative data including zeros?
I have a continuous variable that has many zeros values and is NOT normal, so I can't use parametric statistics ...
2
votes
0answers
191 views
Normalizing data before applying MDS with strain criterion
The features of my dataset are like below:
• BI-RADS assessment: 1 to 5 (ordinal)
• Age: patient's age in years (integer) ...
0
votes
0answers
66 views
Random forests with bagging and range of feature values
Is it important to scale all the features into a common range (normalized) when using random forests (bagging) in classification. Or can random forests handle features in different ranges without ...
6
votes
3answers
532 views
How and why do normalization and feature scaling work?
I see that lots of machine learning algorithms work better with mean cancellation and covariance equalization. For example, Neural Networks tend to converge faster, and K-Means generally gives better ...
1
vote
1answer
175 views
Normalization of likelihood
If I'm not wrong, likelihood functions are sensitive to the size of the sample, i.e. the larger the sample, the lower the likelihood value. Given a sample $x$ of a random variable $X \sim f(\theta)$, ...
1
vote
1answer
428 views
Can I apply a t-test to normalized data?
I'm conducting biological experiments and have to normalize my raw data to the value of the untreated control in order to analyze them. This means that I have for the untreated sample something like ...
1
vote
1answer
103 views
Normalizing when doing count statistics of patient numbers on different arms of a clinical trial
I'm looking at clinical trial data where there are various numbers of patients on different arms (A, B, C, D). All arms receive a drug that is known to cause a toxicity, but some arms receive an ...
1
vote
1answer
148 views
Comparing the result of a study which has unequal group sizes
I have conducted two user studies and in my studies I didn't have control over the group sizes. In each study users were put in groups and they were asked to perform some group activities. Here is the ...
1
vote
0answers
158 views
Z-scores and normalization in previously transformed data sets
I have two sets of data. The first is multivariate linear regression data consisting of betas and standard errors. It was log(e) transformed before the linear regression model was applied.
The ...
1
vote
0answers
84 views
Collaborative filtering and implicit ratings; normalization?
I want to use the time a user spends viewing an article as an implicit rating of how much the user likes the article. My question is how do I normalize this information across all users. At the ...
