Questions tagged [normalization]

Usually "normalization" means re-expressing data to make values lie within a specified range.

Filter by
Sorted by
Tagged with
-1
votes
0answers
14 views

Problem not showing the difference between the values of two sets of data with normalizing the data (between zero and one) (In fact rescaling) [closed]

I have a question about a data normalization problem (between zero and one) I have two indicators whose data are as follows: A: 1 2 3 4 5 6 7 8 9 10 B: 10 20 30 40 50 60 70 80 90 100 When I ...
0
votes
0answers
24 views

Why isn't ROUGE-N normalized by the number of N-grams in the reference summary?

Note: I'll focus on $ROUGE-1$, but the same holds for $ROUGE-N$. For a machine-produced summary $M$ and a bunch of reference summaries $RefSummaries$, I believe $ROUGE-1$ can be calculated in the ...
1
vote
0answers
22 views

How handle frequency data to make it comparable between different sized samples?

I have gathered data on presence / absence of objects at 4 different locations from a number of studies (12 to be exact). I am interested in knowing how frequently each of these objects are recorded ...
0
votes
0answers
21 views

Should I normalise the dependant variable for CART regression? [duplicate]

I am working through An Introduction to Statistical Learning (ISLR), pdf. At "8.1.1 Regression Trees" on physical page 328, it says: We first remove observations that are missing Salary ...
0
votes
0answers
29 views
+50

Creating a popularity index from multivariate data

I am given data from an ecommerce website with features like product_name, product_category product_link, product_id, free_delivery(1 or 0), price, discount, avg_rating, number of reviews, search_rank,...
0
votes
0answers
36 views

Reverse the max-min normalization of Beta/Gamma-distributed data

I normalized my data using max-min normalization as follows $$X_{normed}=\frac{X-\min(X)}{\max(X)-\min(X)}$$ How can I find the distribution of $X$ given the distribution of $X_{normed}$, if $X_{...
0
votes
0answers
18 views

Multivariate feature Gaussianization

Suppose non-negative feature vectors $X = [x_0, x_1, ..., x_{N-1}]$. Existing methods include: BoxCox Lambert Iterative, rotations Log-median: $$ \hat X[n, p] = \log \left(1 + \frac{X[n, p]}{C \mu[p]...
0
votes
0answers
25 views

Floating Normalization of Experimental Data Sets while Fitting Multiple Models

I have $N$ data sets of unequal cardinality, and I am told we do not treat each data set with a normalization of $1$. Instead we let the the normalization float and fit it as though it were any other ...
1
vote
0answers
24 views

When conducting PCA, is it appropriate to mix normalization types?

When conducting a PCA, is it appropriate to mix normalization methods? I'm doing a (personal) project where I am attempting to create an index from some economic time series: 10-year bond yield, ...
0
votes
0answers
22 views

Back transformation of semi-log linear regression model; $\gamma$ factor to allow for non-normality of the residuals?

I'm reading some literature on a model which I am attempting to replicate, where the output (y) has been log transformed prior to fitting. We must back-transform the regression-predicted outcomes to ...
0
votes
1answer
18 views

Min-Max Scaler for fixed effects regression?

I m currently doing some social science research using panel data to determine the impact of budget cuts on financial vulnerability. I have decided to use a fixed effects model to determine this ...
7
votes
1answer
207 views

Scaling dataset in Random Forest

Scaling a dataset for Random Forest modelling is not necessary. However, if we have already done the scaling and normalization to the dataset, will it impact our Random Forest modelling?
0
votes
0answers
14 views

Is it okay to re-scale values that were standardized before some rows were excluded?

The data I was given is scaled to have mean = 0, sd = 1 (no, I do not have the original data, and no I cannot get it). After receiving the data I excluded half the rows, so obviously the resulting ...
3
votes
1answer
65 views

How can I normalize price history between -1 and 1 while keeping the ratio of price differences to each other the same?

I want to normalize a series of numbers to all be between -1 and 1, but I would like to do it in a way where the relative difference between price elements stays the same, if possible. Example list of ...
0
votes
0answers
9 views

Scaling features and target values

My question might be simple, however I can't find an answer for it anywhere. In case of training a model using only one feature, is it important/useful to scale/normalize features and target values ...
0
votes
0answers
6 views

Should I normalize variables after using Ordinal Encoder

I am working on some data that have some categorical variables with a cardinality of 3500. So, I tried to use OrdinalEncoder to transform this column to a numerical column. After that I applied Kmeans ...
1
vote
1answer
26 views

What happens to the data distribution and results if we calculate z-score of a z-scored data?

The data that I am using is already z-scored and batch normalized. I accidentally calculated the z-score again and then performed further analysis and calculated results. Does it make sense to take z-...
0
votes
0answers
12 views

Why does Lasso Regression only eliminate some features [duplicate]

How does Lasso Regression select which coefficients to set to 0 and why are not all of them set to zero? My understanding is minimizing the function: $$ min_{\beta} \lvert\lvert y-X\beta\rvert\rvert^{...
0
votes
1answer
20 views

Using standardized values (z-score) for MAE (Mean absolute error)

I have two models/indices that try to predict observed values. I've compared them using correlation and regression, but I'd like to use MAE (Mean absolute error) to asses which of them is more closer ...
1
vote
1answer
26 views

Normalizing data for better interpretation of results?

I have a dataset: Days compound-1 compound-1rep compound-2 compound-2rep 0 133.77 136.11 3.86 3.91 50 44.26 45.92 1.33 1.21 100 39.71 41.75 0.29 0.34 150 46.23 48.31 1.62 1.71 200 22.11 24.02 1....
1
vote
0answers
24 views

How do you normalize values to 0-3? [duplicate]

I'm aware of course of how you normalize values by using this formula (value-min)/(max-min) and that will give you a range from +1 to 0 but my question is how do you take a value from +100 to -100 ...
1
vote
0answers
15 views

Normalization, centering and PCA [duplicate]

I have a feature matrix composed of frequency responses (in dB) from individual acoustic events. Frequencies in the columns, events in the rows and the matrix is the response The responses decrease ...
0
votes
0answers
14 views

Image normalization with scaled pixel values

I was going through my code and I realized that I first scale values of the image to <0;1> before calculating the channel-wise mean and std. Later down the line, I normalize the images channel-...
0
votes
0answers
9 views

What does "expanded to the mean, max, min, diff and relative diff" mean?

I'm trying to work with a clinical dataset, where lab values and such don´t appear as they are measured, but this way: ...
0
votes
0answers
17 views

ANOVA and normalization to control

I am testing the influence of a categorical variable with several groups on a continuous measurement. The experiment is independently repeated three times (N = 3); each repeat consists of a decently ...
2
votes
0answers
17 views

More Tractable Approach To Causal Path EDA Than Exhaustive Posterior Probability of Models?

I've a dataset of 310 ecological variables aggregated at the State level (and normalized to per capita, etc.) on which I would like to do a causal path exploratory data analysis. In an initial, ...
0
votes
0answers
13 views

Time-series standardization and forecast on original scale

I have a daily time-series data for the number of user registrations on a platform. The series is non-stationary, and it seems not constant in variance even after using the log transformation or ...
0
votes
0answers
12 views

Do I need to normalise/standardise my data, or is percentage change ok?

I'm doing some analysis of two data series, both of which are currency (GBP) over time. My hypothesis is that there's a relationship between the two series. One series is aggregated spend data, the ...
0
votes
0answers
17 views

Should data normalization be applied to the y labels or only to the x data, and why? [duplicate]

I saw places saying that when using a NN you should SOMETIMES normalize your data: we take the X_train, subtract the mean and divide by the std. Is the following ...
1
vote
1answer
55 views

Why do we normalize only the X data and not Y

I saw places saying that when using a NN you should SOMETIMES normalize your data: we take the X_train, subtract the mean and divide by the std. Is the following ...
1
vote
0answers
17 views

Renormalizing a probability matrix in a guessing game

I have a game that's finding the correct one-to-one mapping between members of one group $A$ to members of another group $B$. I have chosen to represent the mapping probabilities as a matrix $P$ of ...
1
vote
1answer
14 views

When is it right to use the log of responses in a relative potency test?

We have two preparations, standard and test described. There is evidence of heteroscedasticity, and less than ten units(unequal for both test). I learnt that assuming normal distribution, gives better ...
0
votes
0answers
15 views

Function to sort a static list of items by recency and frequency

I'm working on a problem which requires me to sort a list of static items for each user. I understand best way to solve this problem would be to come up with a function that captures both the ...
0
votes
1answer
46 views

Normalizing/Scaling a dataset does not have any effect on r2 score?

I have this question on my mind for some time now, but unable to find some thorough explanation around this. While working on the Boston housing data set, scaling the data has no effect on the output. ...
0
votes
1answer
40 views

What is 'Soft' normalization ? (not softmax)

While reading the neuroscience paper "Neural population dynamics during reaching" by Churchland et al. 2012, Nature, the authors mention using 'soft' normalization of their (biological) ...
0
votes
0answers
6 views

Normalizing a custom Weight Shifted or Spiked Gaussian distribution

I have a custom weight shifted bivariate gaussian distribution that I wish to normalize. W is the weighted symmetric matrix that shifts the entire distribution and the λ below is the diagonal matrix ...
1
vote
0answers
19 views

Normalizing product of features

Lets say we have two features x1 and x2 . lets say we do following: x1: Min Max scaling , x2: Min Max scaling x1: Min Max scaling , x2: Z score scaling x1: Z score scaling , x2: Z score scaling Now, ...
3
votes
1answer
48 views

Underperforming SELUs - How to correctly constrain layer weights in TF/Keras?

The promise of SELUs and SNNs I first read up about the 'power' of SELUs on a machine learning blog post. The promise of a Self-normalizing Neural Network (SNN) ...
0
votes
0answers
28 views

Do I need to normalize data lying already between 0 and 1?

I came across a problem in which I have to build a composite indicator for a bunch of similarity measures. All similarities lie already between 0 and 1 and I make no distributional assumptions, as the ...
0
votes
0answers
20 views

How to account for different time periods when trying to calculate the effectiveness of loyalty program

I have the following situation: I would be interested to statistically determine whether or not there are differences between customers who are part of a loyalty program vs. those who are not part of ...
0
votes
0answers
20 views

Should you consider all columns to find normality?

I am new in Machine Learning so please excuse me of my limited knowledge. I am currently working on to learn more on normalization and standardization of data as I understand it is an essential step ...
0
votes
0answers
10 views

Regression - How to control for different ranges in multiple groups

I have a dataset of salary and demographic data. I want to perform a regression analysis to determine if demographic factors (age, sex, orientation etc) influence an individual's salary. Within this ...
1
vote
0answers
42 views

Normalization PCA for dataframes?

I would like to know what is the correct way to normalize a dataframe before applying PCA. I have found two options and I got different results for each one: ...
3
votes
1answer
64 views

Standardization by subtracting the minimum value instead of the mean

A common practice to sum variables that are on different scales is to standardize them, by subtracting the mean and dividing by the standard deviation. However, this produces negative values, which is ...
0
votes
0answers
9 views

Z-score or MinMax for pitch comparison?

I'm using the differences in pitch between two speakers as a feature in a random forest. Given that the corpus of conversations I am using are a combination of mixed and same-sex conversations, I wish ...
0
votes
0answers
52 views

Standardization of index components without mean centering

There are many questions and answers (see here for example) related to standandardization of variables, carried out by taking a value, subtracting the mean (centering) and dividing by standard ...
0
votes
0answers
10 views

Finding a better method for normalising the difference time series of two time series

I am currently working with some sunspot area data available from the Royal Greenwich Observatory's website. They provide daily data from May 1874 to October 2016, which I have used to generate a ...
0
votes
0answers
22 views

Normalization on datasets with different distribution

I am having two datasets one is used for training a model and another one for testing it. The training dataset is large scale corpus of general context (parallel text) while the testing dataset ...
5
votes
1answer
109 views

Self-normalized weighted sum of random variables

Given $n$ i.i.d. random variables $X_1, \dots, X_n$ with $X_i \sim \mathcal{N}(0,1)$ and weights $a_1, \dots, a_n \in [-1, 1]$ such that $$ Y = \frac{P}{Q} = \frac{\sum_{i = 1}^{n} a_i X_i }{\sum_{i =...
0
votes
0answers
28 views

Scaled test data value range is much different than the scaled train data

I am trying to create an LSTM model for time series prediction. I am using MinMaxScaler (from the library sklearn for python) for scaling the data. At first, I didn't know that you shouldn't fit the ...

1
2 3 4 5
23