A way of re-expressing data to make their values lie within a specified range.

learn more… | top users | synonyms (1)

0
votes
1answer
11 views

remove the mean over multiple measurements

I have a set of multiple measurements for each subject (i.e. each subject is assessed several days). For each set of measurements (several days of the same subject) I am calculating the mean value of ...
1
vote
1answer
37 views

Differences between two normalization approaches

I am currently try to normalize data. But I am not sure the differences between $(x - \mu)/ \sigma$ and $x/\sigma$. What are the advantages and differences of these two approaches?
0
votes
0answers
13 views

optimum degree of normalization in gradient descent

I am using steepest gradient approach for solving my my problem and I am using normalize gradient with using second parameter (I am using two parameters so for dimension match I am normalizing the ...
1
vote
0answers
23 views

Google Trends: Stitching 90 day periods of daily data together

Google Trends lets you see the amount of researches made on for a term on google during a set period of time, normalized between 0 and 100 (depending on the highest value in that period). I would like ...
0
votes
0answers
24 views

how to normalize demand/availability matrix for Citibike data

I am not a statistician but would appreciate an outside perspective on my current project analyzing citibike data. This is a bit complicated so please bear with me. My goal is to determine to what ...
0
votes
1answer
22 views

What statistical tool can be used to correct for differences in the amount of data an individual is evaluated on?

Let's say an individual gets a score (between 1 and 6) on different pieces of equipment in their department. For example, if I'm proficient at repairing a particular piece of equipment I will score ...
1
vote
0answers
18 views

Should categorical data be normalised in linear regression?

I have data similar to the following: [ [0, 4, 15] [0, 3, 7] [1, 5, 9] [2, 4, 15] ] I used One Hot Encoder to preprocess this data so it is suitable ...
0
votes
1answer
49 views

data normalization after dimension reduction for classification

The classifier is KNN or RBF-SVM. After doing dimension reduction (e.g., PCA, LDA or KPCA, KLDA), does it need to do normalization before classification? In LIBSVM ...
0
votes
0answers
50 views

normalization to zero mean and variance one logistic regression & random forrests

i was just thinking how does normalization to 0 mean and variance 1 (using an affine linear mapping) can impact the classification accuracy and the choice of hyperparameters when using: logistic ...
0
votes
1answer
23 views

Is it necessary to normalize data for hierarchical clustering of mixed variables using complete linkage?

I have a dataset with 3 numerical variables and 1 categorical variable which is binary (0,1). For clustering these data, should I normalize my numerical variables to the unit range (0,1) by ...
1
vote
1answer
30 views

Exponential family: examples where scaling constant is data dependent

The general form of a exponential family distribution is given as $$p(x|\theta) = h(x) g(\theta) \exp(\theta^Tu(x))$$ where $h(x)$ is referred to as the "scaling constant" (e.g. in Murphy's ML ...
0
votes
0answers
13 views

Error measure in regression task by using a neural network

I am working on a regression task in which I which I want to predict vectors of about 30 values starting from textual documents using a Convolutional Neural Network. In particular, for each document ...
0
votes
0answers
27 views

Normalise heart rate data

I need to statistically examine time series of heart rate data over many different users. Since I don't have data for each user for each instance, I should find a way to normalise the data so that I ...
3
votes
1answer
59 views

Input Normalisation for ReLU neurons

According to LeCun (1998) it is good practice to normalise all inputs so that they are centred around 0 and lie within the range of the maximum second derivative. So for example we would use ...
1
vote
0answers
14 views

is normalization of images of objects centred against a white background reasonable before training a network?

I am trying to train a network to classify grayscale images of two classes say oranges and apples, the apples and oranges are centred against a white background, Before training i am doing contrast ...
0
votes
0answers
26 views

How to use RMSE when having data normalization?

I am new in machine learning and I am studying time series prediction using neural networks. Pseudocode 1: ...
1
vote
0answers
22 views

Calculate power of a test - solution verification

Let $X_1,X_2,\cdots,X_n$ simple random sample from gaussian distribution $\mathcal{N}(m,4)$. Calculate power of a test ($1-\beta$) for $H_0:m=0$, $H_1:m=2$, when significance level of a test is ...
1
vote
0answers
6 views

3 biological groups, 30 subjects (total), measured on 10 days - how to correct for day-to-day variation?

We've run an experiment in which we included 90 subjects and have measured several variables. My question specifically goes to the way we can correct for day-to-day variations. So, here is the setup ...
0
votes
0answers
15 views

Reducing the co-variance of data for denoising, with minimum change in the mean

Assume $X$ and $Y$ are two sets of $n$ dimensional feature vectors from two different multivariate Gaussian distributions with the covariance of $\Sigma_X$ , $\Sigma_Y$ and the mean of $\mu_X$, ...
0
votes
0answers
42 views

Plotting many groups tested with many experiments

I have 1 control group and 9 treatment groups. The 10 groups are tested against 13 treatments (with each have 3 variations so a total of 39) and results are collected. How to plot the results in as ...
0
votes
0answers
10 views

Normalizing factor for distance metric as dimensionality (marginally) increases

The expectation of a squared Euclidian distance from a $d$-dimensional distribution with covariance matrix $\mathbf{I}$ should be $d$, because the expectation of the squared variables making up the ...
0
votes
0answers
19 views

Loopy belief propagation on non-factor graph

I implemented loopy belief algorithm on a graph that is not a factor graph, but I cannot figure out how to normalize the messages so that I do not end up with underflow. What is the correct way to ...
10
votes
4answers
834 views

Is it a good practice to always scale/normalize data for machine learning?

My understanding is that when some features have different ranges in their values (for example, imagine one feature being the age of a person and another one being their salary in USD) will affect ...
1
vote
2answers
93 views

Should I ever standardise/normalise the target data/ dependent variables in regression models?

After standardising the explanatory variables the difference in magnitude between the explanatory variables and the target data is ~3 orders of magnitudes. I want to know if transformation of the ...
0
votes
0answers
12 views

Creating a simple benchmark score system

it's been a long while since I've done any stats so forgive my simple question. I'm trying to do something that my intuition tells me requires statistical treatment, but I don't have sufficient ...
0
votes
0answers
28 views

Constraining and normalizing a function simultaneously

Suppose I have a function $f = f(x,y)$ that I want to constrain so that it is always between $f_\min$ and $f_\max$. I also want to normalize it so that its average is given (i.e., $\int_A f \ dA = ...
1
vote
0answers
18 views

How to demonstrate a tradeoff between two desirable but incompatible metrics/features?

I'm trying to understand a tradeoff that came up to me when I'm studying an algorithm used to distribute a load in a distributed system. The tradeoff involves network communication and make span. ...
0
votes
0answers
22 views

Normalize vs Transform

I am working will pollination data and have quite a few variables and few of them are not normally distributed. I tried transforming them yet few again are not normal. In that case I was suggested to ...
1
vote
0answers
47 views

Use of z-score to normalize two different datasets (price and volume) for comparison

Background: I'm trying to look at the correlation between two datasets, one consisting of price and the other of volume, for a commodity over a period of time. The data are not normally distributed. ...
0
votes
1answer
17 views

reference for valid rates, ratios, normalization, etc

I'm compiling a list of valid ratios that can be used in analysis. For example, # of sub population / # of population, amount by sub population / number in sub population, etc... I'm having trouble ...
0
votes
1answer
56 views

How to normalize data for varying number of observations?

Image shows sample values: bac- Bacteria, prop- Properties I am studying 300 bacteria sequenced from same sample. They are grouped in 7 groups having varying number of members. e.g. some group has ...
0
votes
0answers
20 views

Statistical Method for Normalization of Data

I collected data using a pre-test instrument to capture baseline emotions (two dimensions: valence and arousal) of participants. The two dimensions in the questionnaire range from calm to excited ...
0
votes
1answer
28 views

Normalization on new test set

Many machine learning algorithms require normalization as a preprocessing step. For instance, SVM requires the input data to be normalized. So we do the normalization on the input data and then ...
1
vote
1answer
94 views

Feature scaling (normalization) in multiple regression analysis with normal equation method?

I am doing linear regression with multiple features. I decided to use normal equation method to find coefficients of linear model. If we use gradient descent for linear regression with multiple ...
5
votes
1answer
81 views

Normalisation of circular statistics, such as wind direction in degrees, for clustering

I have a set of data points each representing a day and a number of features associated with it: temperature, wind speed, wind direction, humidity... etc. Before the analysis, I am meant to normalise ...
0
votes
1answer
22 views

Normalizing a large set of ratios

I have a problem that has been causing headaches. To set up the problem, here's what I have. Bear with me, since I'm trying to clarify the problem as much as possible: A large set of personnel These ...
0
votes
0answers
16 views

Normalizing logarithmic data in a range from 0 to 1

i am replicating a multilevel-analysis for my bachelor thesis (with newer data + corrections). The author i am citing used the unstandardized coefficients of every independent variable as beta ...
0
votes
0answers
15 views

Strange behaviour of energy in CNN

I was using the examples in the tutorial here, then I wanted to adapt it to another dataset, so I collected the same data struct (as imdb), and I decided to ...
0
votes
1answer
22 views

Judgement score regularization problem

Consider the scenario where M performances (eg. singing contest) are being judged by N judges. Each judge awards a score S(m,n) to each performance on a scale of one to one-hundred. The problem ...
0
votes
0answers
27 views

Proper way to do multivariate analysis of relative abundance data table

I have a data table, which is a result of running software MetaPhlAn. The values are relative abundances of each microbe taxon in samples. Samples are independent of each other. I want to know how ...
0
votes
0answers
20 views

Normalising a range of values into another range of values

I want to normalise a set of range of values having 0 Min and a Max that is known but can vary; say 22000 and would like to normalise these values from 0 to 300 and also from 0 to 20. I found a ...
0
votes
0answers
33 views

95% confidence interval of ratio of two independent variables

I have two unknown random variables W and A, where W~(nx,ny) and A~(x,z). My aim is the find out the range of n for W/A. I tried to first standardised W and A and make it greater than -1.96 and small ...
0
votes
0answers
19 views

Normalized back theta after linear regression

This is how I work : 1- I get data 2- I scale those data : ...
0
votes
0answers
17 views

Calculate distances of a company branches from main branch in every state. How calculate distance for main branch in every state?

Suppose that I'm calculating distances of a company branches in every state from main branch of that state. After that I'm combining this feature with other features to creating a composite indicator ...
0
votes
1answer
46 views

Is it possible to normalize data by different group leaders separately?

I have a dataset that contains different states of a country. In every state there are different companies and one company in every state is manager of other companies in that state (other companies ...
1
vote
1answer
46 views

Standardizing or normalizing count data

I have a dataset where I counted the number of a species in different environments and grouped it into different categories ranging from 0 to 5. 0= no occurrence; 5= very high occurrence. All ...
0
votes
1answer
50 views

Can I use one way ANOVA for my normalized data?

I have trouble knowing the right statistical test that i need to use for my data. I have a group of animals subjected to a task, where each animal is exposed to three or four conditions. The data ...
2
votes
2answers
42 views

Support Vector Regression and Data Rescaling

I am currently working on Support Vector Regression and I've read that it is recommended to implement data rescaling, e.g. to interval $[-1;1]$, to obtain better results. My first question is: should ...
0
votes
0answers
5 views

OSC-filter is a also a kind of normalization?

Before to aplicate an OSC filter (in SIMCA-P software) is necesary to have normalized my data? or does no matter? Even more, what is the effect of applicate log transform and a Pareto scale after OSC ...
0
votes
0answers
8 views

Is there a danger in limiting output values in a small range?

I'm implementing an algorithm that estimates the 2-D distances of $P$ pairs of points on a dataset of $N$ images, via ridge regression (so its estimation $y$ is $\in \mathbb{R}^{2P}$). I was doing ...