A way of re-expressing data to make their values lie between 0 and 1 (or 0% and 100%).

learn more… | top users | synonyms (1)

1
vote
0answers
13 views

How to normalize data in close range 0.95-1.05 to scale between 0.9 and 1? [on hold]

everybody, I have a problem: I have data in close range (0.95 to 1.05 or -0.05 to to 0.05), but these values (presented as percentage in the end) can not be presented as negative numbers nor with ...
0
votes
1answer
48 views

Normalize variables for calculation of correlation coefficient

I have two vectors (arrays) of values. One vector represents a variable whose values are between 0 and 1 (ratio-type variable). The other vector represents a variable whose values are continuous float ...
0
votes
0answers
12 views

Normalisation formula applicable to one or more data items

I've created a recurrent neural network, to which normalised values are passed as inputs. The normalization formula is: $$\tilde{x_{i}} = \frac{1}{1+exp(-\frac{x_i-\bar{x}}{\sigma})},$$ where ...
0
votes
0answers
34 views

How to determine a data transformation factor

I'm working on transforming one set of data to another based on a certain variable (length). Here's how the actual problem is like: ...
0
votes
0answers
23 views

Normalization in case of clustering

Can normalization in this form be used (x−μ)/σ and should it be used in case of clustering? I have parameters on different scales and since I'm calculating the distances I need to perform feature ...
0
votes
1answer
25 views

Normalization problems - how to normalize in case of set of points while new points arriving

I'm having a procedure in which I perform clustering, and later, for each new example I test if that example belongs to some of existing clusters, by calculating distance to existing centroids. To ...
1
vote
0answers
35 views

How to normalize the data to [0, 1] in R with data similar to χ²-distribution without shrinking lower values too much?

I want to normalize the data to [0,1], but the distribution of this array is quite not regular, having large quantity of low values and small quantity of large values, almost 80% values of data are in ...
1
vote
1answer
29 views

Noise removal from a dataset with a know distribution

If I have a dataset where it's points are drawn from a known distribution(For example a normal distribution) due to some noise the histogram doesn't reflect such a behavior (not necessarily skewed) ...
0
votes
0answers
7 views

Quantile normalize a single column in R [migrated]

I have a column in my dataframe in R, data$height. The values range from 0- 400. I want to normalize the values in the column such the resultant values lie between 0-1 and are quantiles, i.e the ...
2
votes
2answers
37 views

Why do we whiten the data before running ICA?

Why do we usually pre-whiten the data before doing independent components analysis (ICA), like making all eigenvalues equal? Doesn't that take away some information regarding the data?
0
votes
0answers
13 views

r-pnn, normalization and different distance measures for each variable

Since pnn is a NN that uses a Radial kernel to classify data, I think the distance measure is key and, in consequence, the normalization of the data. Am I right? How does pnn package calculate the ...
0
votes
1answer
57 views

When normalization is counter-productive [duplicate]

Could you give me general examples of when normalization is not used properly and affects badly the classification accuracy, or when it is not needed?
0
votes
0answers
9 views

System failure data - normalize data when there is more data available for that system

Suppose I have the following data from 6 systems. The number of Type A systems is 2 in 6 so 1/3. There are 2/3 of B systems. In the 6 systems there were 40 failures in total. ...
4
votes
3answers
233 views

When should I apply feature scaling for my data

I started a discussion with a collague of mine and we started to wonder, when should one apply feature normalization / scaling to the data? Lets say that we have a set of features with some of the ...
0
votes
0answers
18 views

Which type of feature scaling to use [duplicate]

I've been writing some simple machine learning algorithms and looking at time-series data. In doing so, I have come across the use of feature scaling: rescaling and standardizing as they are referred ...
0
votes
1answer
34 views

Question in Artificial neural network normalization

I wish to normalize inputs parameters into [0 - 1] and fit into a neural network for training. I have done a simple normalization method - MinMax Question 1: if my inputs have negative values, do i ...
1
vote
1answer
36 views

Cumulative standard deviation of variables with different distribution

Suppose we have a set of basketball players, each of which have 9 associated performance categories. Each of these categories has a different distribution. I want to find a good way to represent how ...
2
votes
1answer
23 views

Symbol to indicate normalized or standardized variables

Is there a symbol to indicate that variables have been standardized? For example, if I have 2 different scoring functions Score1 and Score2. Let's say I want to form a combo score and show that the ...
0
votes
0answers
14 views

How should I normalize a difference of features?

Say I have two Feature vectors $\phi_1$ and $\phi_2$. Theoretically which of the following normalization alternatives would give greater average performance for a binary classification problem that ...
1
vote
1answer
44 views

Data normalization in k-means and svm

Generally if I want to normalize my data in which direction I should normalize (subtracting mean and dividing by std)? Lets say I have a data matrix D (...
1
vote
1answer
40 views

How to decide how to normalize a given feature in a data set?

I have two features that differ in their orders of magnitude and also in the way they are distributed. I am aware of two ways of normalization: 1) $\frac{x - x_{min}}{x_{max} - x_{min}}$ 2) $\frac{x ...
0
votes
1answer
43 views

Why do we need undirected (Markov) graphical models?

I understand the modular nature of directed models, and that each node captures a conditional probability. But why do we need undirected models? As far as I can see they lack intuition in that the ...
0
votes
0answers
12 views

real time object tracking

I have correlation value of object using mean and standard deviation, now i want to normalized that correlation value. how can get normalized correlation in terms of percentage vale?
4
votes
1answer
64 views

If I divide my data by its mean, does it still have a unit?

When dividing a timeseries by its mean value so that its mean becomes 1, does the resulting data still have a unit or is it unitless?
0
votes
0answers
25 views

UnReSolved Mean Adjust DataSet to achieve .5 Mean [duplicate]

Update So I've done some of my own work on transformational methods, and the best I can get is what I call an s transform as detailed in this workbook; however, various attempts at trying to mean ...
1
vote
2answers
70 views

Scaling a series of numbers

Suppose I have 30 numbers that vary between 0 and 1.0 and which sum to 1.0. The mean is obviously 0.033. A client wants these scaled to lie between 0 and 1.0 but to have a mean of 0.5. By the way, ...
0
votes
0answers
12 views

Normalize sequence lengths

I'm implementing a Markov Chain Model for web site access sequences. Some sequences are very lengthy and some are very short containing a single transition. Lengthy sequences generate a very low ...
2
votes
1answer
55 views

Can we run a chi squared test on a normalized function?

Hi I am fit a maxwell distribution and attempt to find the chi squared value in two cases: When the data is normalized. When the data is un-normalized. My problem is that the two ...
0
votes
0answers
12 views

Assign attributes / categories to users based on their activity / likes

I have a very practical classification problem for which I need some help. I have a database of users along with their activity / likes for a number of car models. I also have the category each of ...
0
votes
0answers
22 views

Most efficient vector construction for Dynamic Time Warping

I'm in the process of folding FastDTW into my SVM and the question now is how to best format my data (irrespective of normalization). Here's an example of what I'm attempting to do - given two 3d ...
1
vote
0answers
38 views

Order of preprocessing steps in a binary classification problem

I have these stages (ordered) for preprocessing in my binary classification problem. Dividing data based on criteria (class1 and class2 databases) Outlier ...
0
votes
0answers
51 views

normalizing a dataset

I have a dataset that has budget numbers for various organizations. The numbers range from less than a million to hundreds of millions. The dataset also has information about various departments in ...
0
votes
0answers
24 views

Proper “normalization” of post-test data to pre-test conditions

I'm performing an in vitro assay in which cells are treated with different combinations of two different compounds and the activity of the cells following treatment is measured. There are 3 ...
1
vote
0answers
36 views

normalizing a proportion of poll results given unequal gender response rates

I took a poll of patients who experienced at least one misdiagnosis. I'd like to analyze the data according to gender but 75% of the responses came from Females and 25% from Males. Example: The data ...
0
votes
0answers
14 views

newbie question - regression vs composite normalised scores

Lets say I need to rank the "strength" of a bunch of objects based on a number of factors, as far as I can tell, there are two ways people seem to commonly do this : (a) calculate normalised scores ...
1
vote
0answers
17 views

Un-smoothing/scaling (help normalizing data)

The context of this is matching experimental RNA SHAPE data to theoretical models of base pairing probabilities. RNA folds back on itself, some bases pair with each other, and some bases remain ...
2
votes
1answer
110 views

Standardizing before/after/at all when using multi-class LDA for pre-processing step

If a multi-class Linear Discriminant Analysis (or I also read Multiple Discriminant Analysis sometimes) is used for dimensionality reduction (or transformation after dimensionality reduction via PCA), ...
1
vote
1answer
78 views

Normalizing Vs. Scaling

Are the concepts of normalizing and scaling of data in conflict with each other? I am adding weights to my features, I have tried normalizing the weights and it didn't make any difference in the ...
0
votes
1answer
71 views

How to : a brief intro to scaling and rescaling data ( inputs) for supervised learning algorithms

I understand the concept of scaling and that it improves results in SVM's and NN's. however I would like to find somewhere where is is explained, in easy "layman's terms" terms. of how it is done. I ...
1
vote
1answer
37 views

Scaling in SVM (why and how to , plus references)

Hi I know why feature scaling is preferred in SVM, I have two questions: 1-does anyone know of legit articles of books explaining it. I am writing my thesis and I need references. It doesnt have to be ...
2
votes
0answers
59 views

Averaging z scores when doing a “group by”

I have a dataset where each row is an hourly measurement of certain fields (columns). For each column I then add another column that is its respective z score relative to the entire population. If I ...
1
vote
0answers
103 views

How to normalize bimodal (or multimodal) distributions?

If I have multiple data series, a = [a1, a2, ... a100] ~ bimodal with mu_a1, mu_a2, sigma_a1, sigma_a2, b = [b1, b2, ... b100] ~ bimodal with mu_b1, mu_b2, sigma_b1, sigma_b2, c = [c1, c2, ... ...
0
votes
0answers
33 views

Usefulness of Z-normalization in Machine Learning

Z-normalization means rescaling the feature $X$ by subtracting the average $\mu$ and dividing by its standard deviation $\sigma$, i.e., $(X-\mu)/\sigma$. What is the usefulness of normalizing data ...
0
votes
1answer
21 views

What is the range of values that can be expected in the result of Principal Component Analysis (PCA)?

I want to normalize all of my preprocessing techniques between 0 and 1 so I want to know what the PCA range of values is so that I can apply a proper normalization to it. I applied PCA by using the ...
1
vote
1answer
37 views

Are SVD (Singular Value Decomposition) values always positive? Is there a relation between the maximum SVD value and the original data?

Assuming it's the standard SVD (no variation of it) with $A = USV^T$, would the $A$ matrix always have positive values (0 to $\infty$)? I noticed that the $U$ and $V^T$ matrices had some negative ...
0
votes
0answers
24 views

Normalization against Covariates

I have a list of parameters which correlate with 1-2 covariates that I want to control for. Following normalization, I wanted to do comparisons between groups, correlation analysis and probably use ...
0
votes
1answer
97 views

Using F-tests for variance in non-normal populations

I'm fairly new to stats, so please excuse me if this problem is hopelessly elementary or misinformed. Basically, I'm wondering if you can help me understand whether I'm using the F-Test for variance ...
0
votes
1answer
66 views

Feature Normalization/Standardization before or after Feature Selection?

Should the process of feature normalization/standardization be done before or after the feature selection process?
0
votes
1answer
23 views

Does a time-series have to be stationary before you calculate a z score or t score?

It's been a long time since basic statistics. I have a financial time-series that exhibits exponential growth. Before I standardize, do I have to make the time-series stationary? Before I ...
3
votes
1answer
118 views

Inputs to k-means are often normalized per-feature. Why not fully whiten the data instead?

We often normalize inputs to the k-means algorithm by 1) subtracting the mean on a per-feature basis and 2) dividing by the standard deviation on a per-feature basis. Some rational behind this is ...