A way of re-expressing data to make their values lie between 0 and 1 (or 0% and 100%).

learn more… | top users | synonyms (1)

0
votes
0answers
15 views

Standardizing before/after/at all when using multi-class LDA for pre-processing step

If a multi-class Linear Discriminant Analysis (or I also read Multiple Discriminant Analysis sometimes) is used for dimensionality reduction (or transformation after dimensionality reduction via PCA), ...
1
vote
1answer
36 views

Normalizing Vs. Scaling

Are the concepts of normalizing and scaling of data in conflict with each other? I am adding weights to my features, I have tried normalizing the weights and it didn't make any difference in the ...
0
votes
1answer
24 views

How to : a brief intro to scaling and rescaling data ( inputs) for supervised learning algorithms

I understand the concept of scaling and that it improves results in SVM's and NN's. however I would like to find somewhere where is is explained, in easy "layman's terms" terms. of how it is done. I ...
1
vote
1answer
22 views

Scaling in SVM (why and how to , plus references)

Hi I know why feature scaling is preferred in SVM, I have two questions: 1-does anyone know of legit articles of books explaining it. I am writing my thesis and I need references. It doesnt have to be ...
2
votes
0answers
29 views

Averaging z scores when doing a “group by”

I have a dataset where each row is an hourly measurement of certain fields (columns). For each column I then add another column that is its respective z score relative to the entire population. If I ...
1
vote
0answers
39 views

How to normalize bimodal (or multimodal) distributions?

If I have multiple data series, a = [a1, a2, ... a100] ~ bimodal with mu_a1, mu_a2, sigma_a1, sigma_a2, b = [b1, b2, ... b100] ~ bimodal with mu_b1, mu_b2, sigma_b1, sigma_b2, c = [c1, c2, ... ...
0
votes
0answers
25 views

Usefulness of Z-normalization in Machine Learning

Z-normalization means rescaling the feature $X$ by subtracting the average $\mu$ and dividing by its standard deviation $\sigma$, i.e., $(X-\mu)/\sigma$. What is the usefulness of normalizing data ...
0
votes
1answer
18 views

What is the range of values that can be expected in the result of Principal Component Analysis (PCA)?

I want to normalize all of my preprocessing techniques between 0 and 1 so I want to know what the PCA range of values is so that I can apply a proper normalization to it. I applied PCA by using the ...
1
vote
1answer
21 views

Are SVD (Singular Value Decomposition) values always positive? Is there a relation between the maximum SVD value and the original data?

Assuming it's the standard SVD (no variation of it) with $A = USV^T$, would the $A$ matrix always have positive values (0 to $\infty$)? I noticed that the $U$ and $V^T$ matrices had some negative ...
0
votes
0answers
14 views

Normalization against Covariates

I have a list of parameters which correlate with 1-2 covariates that I want to control for. Following normalization, I wanted to do comparisons between groups, correlation analysis and probably use ...
0
votes
1answer
41 views

Using F-tests for variance in non-normal populations

I'm fairly new to stats, so please excuse me if this problem is hopelessly elementary or misinformed. Basically, I'm wondering if you can help me understand whether I'm using the F-Test for variance ...
0
votes
1answer
33 views

Feature Normalization/Standardization before or after Feature Selection?

Should the process of feature normalization/standardization be done before or after the feature selection process?
0
votes
1answer
16 views

Does a time-series have to be stationary before you calculate a z score or t score?

It's been a long time since basic statistics. I have a financial time-series that exhibits exponential growth. Before I standardize, do I have to make the time-series stationary? Before I ...
3
votes
1answer
58 views

Inputs to k-means are often normalized per-feature. Why not fully whiten the data instead?

We often normalize inputs to the k-means algorithm by 1) subtracting the mean on a per-feature basis and 2) dividing by the standard deviation on a per-feature basis. Some rational behind this is ...
0
votes
0answers
14 views

Normal Data Distribution (Visual tests accept and statistical test ks reject normal distribution) Need Help [duplicate]

I have four variables want to run regression while i check for data distribution i found that histogram and qq plot provide evidance of normal data distribution where as ks test is significant for all ...
3
votes
1answer
47 views

Is there a formal name for this data normalization formula?

I am using a generalized formula for normalizing one data range to another but am having difficulty finding its formal name, if it even exists (sorry if my notation is strange): $$ x_b = min_b + ...
0
votes
0answers
26 views

regression trees + scale of output variable

I am developing a regression tree model I have an output variable with a very large standard deviation, I am wondering if I need to scale/normalize this output variable as metrics such as RMSE and R^2 ...
3
votes
1answer
47 views

Difference in tf-idf values in R

I am playing around in R to find the tf-idf values. I have a set of documents like: ...
2
votes
1answer
33 views

Normalization with error in denominator

I am trying to come up with a proper way to normalize my data. Using a microscope I want to count the percentage of green cells in a population. However, only ~0.1% of all cells are green. I decided I ...
0
votes
1answer
34 views

Data normalization with preference to a number size

I understand that data normalization allows us to take data and place it on a scale of [0,1]. Currently I'm working through a machine learning book and the author talks about normalizing data with ...
2
votes
1answer
30 views

standardising sub-data sets, such that the whole dataset is standardised

Consider a data set $X$ made up of smaller subsets: $X=A \cup B \cup C$, with $A,B,C$ disjoint data sets. Eg: $A=\{1.0, -1.0, 0\}$, $B=\{5.0, -7.0, 2.0\}$, $C=\{1.5, -5.0, 8.0\}$ Is is possible ...
0
votes
0answers
14 views

How to create a score that gives same weight to N variables using different scales?

I'm analyzing a website, we have three variables Pageviews, Minutes spent on site and Entrances and want to produce a score that gives equal weight to each of them. At first I was simply going to ...
1
vote
0answers
28 views

Normalizing weekly sales fluctuations in the overall market

I have a rudimentary question regarding how to "normalize" a set of time series data, and would appreciate your thoughts. To make it very simple, the hypothetical situation is as follows: Suppose we ...
0
votes
1answer
19 views

Accelerometer normalization

I'm working with data from different accelerometer hardware. Each hardware has a different maximum range, different resolution at which data changes, and different minimum delay after which a new ...
0
votes
0answers
30 views

Normalize by expected value

If I have an underlying distribution of expected values, how do I normalize my observed values by this distribution? Here is an example: I am testing for a deviation from 50:50 (my Null ...
0
votes
1answer
51 views

De normalize predicted value

Alright so i have found this really good answer on how to normalize my data. I implemented @user25658 's code into my own project successfully, trained a linear model and used it to make a ...
1
vote
0answers
96 views

New Systematic Quantile Normalization

I think I've developed a systematic quantile normalization technique. I did this with music but I think it can also be done with light and other frequency based information. The algorithm is as ...
0
votes
1answer
38 views

How to analyse data from different subjects?

DESIGN: I have 4 laboratory mice (= 4 subjects). My factor is a condition with 5 levels ...
0
votes
1answer
70 views

histogram normalised to area 1

i have a histogram with the y-axis showing the proportion in percentage. That makes sense to me but now i have read that histograms can be normalized with the result that the area of the rectangles is ...
0
votes
1answer
66 views

Too big (?) histogram values when using normed histo options in SciPy and matplotlib

I have been trying to create a normed histogram using either SciPy or matplotlib (or anything for Python). When I create my histogram with 'normed' option disabled, it looks like below (this example ...
1
vote
1answer
33 views

When to scale/normalize for supervised learning algorithms?

I'm trying to understand which supervised learning algorithms require normalization/scaling of features. It appears that when an algorithm works by calculating the conditional probability (Naive ...
0
votes
1answer
34 views

Normalization Factor divide or multiply

If I want to normalize some data using a median normalization or trimmed mean normalization, do I multiply or divide my data by the normalization factors? Does it matter?
0
votes
2answers
113 views

Normalization of dummy variables

My data consists of several continuous measurements and some dummy variables representing the years the measurements have been made. Now, I want to learn a neural network with the data. Therefore, I ...
1
vote
1answer
29 views

how to transform data of two experimental groups? one is positively skewed and one is negatively..

I have two experimental groups. Then I test their normality respectively. Result shows that one is positively skewed and the other is negatively skewed. In this case, how should I do the data ...
0
votes
0answers
34 views

Normalizing depth data

I have some Kinect data of somebody standing (reasonably) still and performing sets of punches. I am given it in the format of an x,y,z co-ordinate for each joint of which they are 20, so I have 60 ...
1
vote
2answers
68 views

How to transform negative data to be homoscedastic

I have a bunch of data that's both positive and negative. Its calculated from the residuals of an ANOVA (i.e. specific leaf area calculated as the residuals of an ANOVA of leaf area with leaf blade ...
0
votes
1answer
16 views

Normalization for pattern classification?

I'm working off my first independent project for some pattern classification. I'm utilizing some datasets from UCI machine learning, but am not sure on how to start with data normalization. The data ...
1
vote
1answer
79 views

SVM data normalization… what about classifying new (training) data?

I've got a big doubt about SVM classification task (and more in general classification task), about data normalization. Let's suppose I've a SVM trained with normalized data, and new data to classify. ...
1
vote
0answers
18 views

Pre-stimulus baseline removal in R

I have the following scenario: trials were conducted where participants were exposed to multiple stimuli during the course of a trial a specific physiological response was continuously recorded ...
0
votes
0answers
40 views

Scaling/Normalisation or Standardization

I'm working on SVM and ANN classification tools. In order to improve the classification accuracy, I want to know the best or the recommended data-preprocessing, is it scaling/normalisation or ...
1
vote
0answers
20 views

Normalization on frequently updated dataset

There are some normalization types like rescaling, standart score or modified standart score. I can apply these algorithms to large dataset. If the dataset that i am working on getting frequently ...
1
vote
0answers
24 views

The number of coin tosses needed if the proportion of heads is to lie within 0.05 of p with probability at least 0.9?

There's a question I'm not really sure if I did it right or even understand what its trying to say. There is a coin which produces heads with an unknown probability $p$. How many times should we ...
0
votes
0answers
23 views

corpus linguistics statistics

I am in doubt as to how calculate observed relative frequences in corpus linguistics. That's how I did it: I multiplied the number of words by thousand and then I divided it all by the size of the ...
1
vote
1answer
60 views

Comparison of subpopulations: do I need normalisation?

I have a population of people. Each person has one of three characteristics, say X, Y or Z. I want to compare other characteristics of these people, using the characteristics X, Y and Z as a ...
1
vote
0answers
44 views

Obtain the covariance matrix with the correlations and the variances

i have the following problem that makes me crazy. COnsider a trivariate normal variable, let $\sigma_1^2$, $\sigma_2^2$ and $\sigma_3^2$ be the variance of the three components and $\rho_{1,2}$, ...
2
votes
0answers
45 views

multiple linear regression with normalization - how to get non-scaled full covariance matrix

I am doing a quite complicated multiple regression modelling in physics and have a problem how to got back to covariance matrix for non-normalized parameters. I don't know how to calculate the error ...
0
votes
1answer
70 views

Normalize row or column while each row is an observation

Suppose I have a matrix compose of row as each observation, column as each property and I want to calculate the distance between each observation. In this case I think I should normalize each column, ...
0
votes
1answer
82 views

Normalize histogram with different bin width?

I know how to normalize histogram (so that the area =1) with the same bin width, but how to do it when the histogram has different bin width? Any idea?
1
vote
1answer
68 views

normalizing dataset for extrapolation - sample or population mean and standard deviation?

I am currently fitting models that are intended to be used for extrapolating from a limited sample to a large population. For a specific example, one model is predicting water temperature in rivers ...
0
votes
1answer
92 views

How do i normalize residuals?

I am trying to adjust a hierarchical multiple regression model and no matter which transformations I use (z-transformation, sqrt, cuberoot, inv, inv sqrt ...), I do not manage to get the residuals ...