Skip to main content

Questions tagged [normalization]

Usually "normalization" means re-expressing univariate data to make values lie within a specified range.

Filter by
Sorted by
Tagged with
0 votes
0 answers
18 views

Normalization of absorbance data by fresh weight of samples

A [protocol] says For each sample the absorbance (Abs) reading was divided by the fresh weight of the sample in grams. The results were normalized using an arbitrary value of 1 for the control ...
user414999's user avatar
0 votes
0 answers
10 views

Normalization of time-series data before computing Dynamic Time Warping (DTW)

I am interested in a proper normalization method of my time-series data before computing the dynamic time warping (DTW) distance. Situation: I have four time series (colored in two different green ...
Philipp's user avatar
  • 245
0 votes
0 answers
5 views

Is it wrong to use multiple normalizers sequentially on a dataset?

I am working on runoff prediction. I completed missing data and removed outliers. When it comes to normalization, applying a single normalization method (such as MinMaxScaler, StandardScaler, or Power ...
user412750's user avatar
0 votes
1 answer
46 views

Principal Component Analysis using Panel Data

I have a panel data with identifiers(a,b,c,....z) and different times(t=1,2,3,....100) I have 6 different variables (A,B,C,D,E,F) for every identifier-time observation. I attempt to use those 6 ...
Ken's user avatar
  • 1
0 votes
0 answers
15 views

Target variable standardization for lasso regression

I am working with different models for a regression task. The range of my target variable is very small: I noticed a very bad performance of the lasso regression and elastic net model in comparison ...
Limmi's user avatar
  • 13
0 votes
1 answer
76 views

Simple example of Log-Sum-Exp trick for continuous case

I am trying to confirm my understanding of how to apply the [Log-Sum-Exp trick to recover a posterior distribution from a log-posterior distribution. I want to consider a simple example from a model I ...
John Doe's user avatar
  • 185
0 votes
0 answers
13 views

Normalization of time-series data with time-varying variance

I'm building a neural network (CNN) model for a regression problem with time-series data. Both input and output are multi-variate zero-mean timeseries data with time-varying variance. Currently, I am ...
P. Leibner's user avatar
0 votes
0 answers
21 views

Standardization vs Normalization of data

In the above example I am using 3 methods for distance calculation 1.Calculating distances of actual data 2.Standardizing the data and calculating the distances 3.Using min-max scaler and then ...
Hari Krishna's user avatar
0 votes
0 answers
15 views

Good way to select reference and/or performing normalization of data points across experiments?

I have a set of objects S1 = {a1, b1, c1, d1, ..., z1} that perturbed a reference object q and were tested for effects in ...
neo4k's user avatar
  • 1
0 votes
0 answers
21 views

Can principal components changed by a normalization method be used to construct original data shape with SVD

I'm planning to use an algorithm called Harmony, designed for data normalization, particularly in the context of single cell data analysis. Harmony operates by taking principal components (PCs) as ...
MadmanLee's user avatar
  • 133
0 votes
0 answers
20 views

How can I normalize Hedges' g from two extreme conditions around the control condition?

The variables I am discussing below are not the ones I am actually using but I think that they should give a better sense of what I am trying achieve. I am performing a meta-analysis of some chosen ...
aria's user avatar
  • 1
2 votes
1 answer
107 views

How to normalise outputs of neural networks with different distribution?

I have a NN model that predicts 8 different variables. I use a multi-task learning approach, where I compute the loss between predictions and targets for each of ...
bird's user avatar
  • 95
0 votes
0 answers
29 views

Help interpreting normalized HMM (or otherwise) results

I have run a hidden markov model with five variables on very different scales. Because of this I normalized the input data beforehand using Carets preprocessing: ...
Jcarroll's user avatar
0 votes
1 answer
42 views

Model comparison: with raw or normalized data?

I have developed a index of drug addiction risk whose formula is Index = 1/log10(a_given_variable). The raw values of the calculated ...
M.S.'s user avatar
  • 1
1 vote
0 answers
11 views

Do I need to normalize corpus frequency if I am not comparing between corpora?

Do I need to normalize my corpus frequency, given that I am not comparing corpora? For example, if I am going to compare collocations of lexicons A, B and C in a corpus with 13 million tokens, can I ...
user avatar
0 votes
0 answers
26 views

Normalize age frequency data for PCA

I am working on a project to forecast house ownership rates. One dataset I have consists of number of people of each age from 1-99 per geographic area code. For example, 20 people aged 1, 59 people ...
burn_burn_55's user avatar
2 votes
2 answers
114 views

Controlling for number of days when calculating medication adherence

I am trying to evaluate the success of a medication adherence intervention. I am trying to assess the success of the program before and after the intervention in terms of number of days a patient took ...
NC8389's user avatar
  • 21
0 votes
0 answers
33 views

Versatility and validity of aggregate algorithm for a multiple criteria decision problem (averaging)

I'm trying to aggregate different variables in multiple scales and distributions. The distributions are not known beforehand, but I want to generate a general statistical value that represents all the ...
gabriel's user avatar
  • 93
0 votes
0 answers
67 views

Understanding the Normalization/Standardization of geospatial coordinates

I'm building a neural network to predict future [latitude,longitude,altitude], and am having trouble dealing with the features. I've reviewed the answers to the ...
LivelyECDSA's user avatar
1 vote
0 answers
39 views

Time series normalization

I'm not an expert yet in the field and I have some questions. I have some data of birds and drones taken from a radar. I want to create a classifier that differentiates them. At first I'm trying and ...
Victor's user avatar
  • 11
1 vote
0 answers
23 views

Normalization for time series comparison

I have a time series Markov Switching model, which is estimated in about 15 different versions. One or two of the time series had to be normalized in order to converge. That is 1-2 out of 15. My ...
David K's user avatar
  • 31
0 votes
0 answers
26 views

Normalising series data of variable length

I have a model m which takes as input a starting array of n_input elements which are actually traces of mining processes and ...
Shivam Roy's user avatar
2 votes
1 answer
326 views

What are the layer normalization dimensions in transformer?

In transformer training, the activations have three dimensions: batch, feature (i.e. embedding) and time (i.e. token). Layer normalization is applied, calculating statistics (mean, standard deviation) ...
Denziloe's user avatar
  • 1,143
0 votes
0 answers
15 views

Normalize data with a variance-shift in measurement-error to achieve high correlation with true underlying process?

I am discussing the question whether to normalize the data or not in the following setting: I have a true time series $$s_t = iid(0,\sigma_s^2)$$ however I only observe different $$\hat{s}_{t,i} = s_t ...
marc's user avatar
  • 1
0 votes
0 answers
31 views

Normalization/standardization of time series data

I have energy consumption data where rows represent different users and columns are different measurements. I don't really understand, how and in which order i need to normalize and standardize the ...
Moulmein's user avatar
0 votes
1 answer
16 views

Interpretation of income rank variable

I have an income rank variable which is calculated using the following formula: Income rank = (j-1)/(n-1) where j – 1 represents the number of people within the individuals' comparison group with ...
Eran's user avatar
  • 111
2 votes
1 answer
41 views

How to Normalise a loss function containing actual values?

I defined a loss function that I am trying to optimise consisting of two elements. let's say a and b: w1 * (a_actual - a_measured)^2 + w2* (b_actual - b_measured)^2 I am trying to simplify the ...
user avatar
0 votes
0 answers
41 views

Quantitative Methods for Evaluating Differences Between Two Distributions

I am working with a substantial dataset in which I need to compare the distributions of certain common features across different categories. The challenge I face is that due to the imprecision in ...
Song Nyanko's user avatar
0 votes
0 answers
23 views

Interpreting normalized value in logistic regression

I'm trying to interpret results from my logistic regression. I normalized the values between 0 and 100 because I had many variables with different units. My intercept is -0.741 and I'm using that to ...
imane fouiteh's user avatar
0 votes
0 answers
124 views

Normalization of log-returns or normalization of cumulative log-returns

This questions seeks for discussion to find theoretical support for normalizing cumulative log-returns vs normalizing log-returns By "normalizing" (also known as standarizing) I mean it in ...
Alfonso's user avatar
  • 21
0 votes
1 answer
40 views

Transform data domain and maintain mean

I have a dataset, $y$, that is on some arbitrary range. I would like to transform this data to be on the range [-1,1]. This is accomplished using a linear transformation, such as the one described ...
Colton Campbell's user avatar
0 votes
1 answer
37 views

Data normalisation for a newbie

I am new to data analysis and I have been given a task to compare safety of trams compared to buses. Since there are far more buses than trams, I was introduced to the concept of normalisation by ...
Revonda Sanchez's user avatar
0 votes
0 answers
14 views

How many number of samples must be taken to ensure no information is lost in normalization?

I was using nn.LayerNorm from Pytorch to normalize my dataset. I tested on a tensor of two points but surprisingly found that my output is almost always the same. Going off the equation of ...
Fine-Tuning's user avatar
2 votes
1 answer
57 views

Should I apply normalization to predicted probabilities from 7 different models before computing correlation among them?

I'd like to check if there are correlation among predicted probabilities of models in a voting classifier. According to the table below, one of models, Model5, has mean 40.9% and standard deviation 46....
KAI's user avatar
  • 23
2 votes
1 answer
58 views

Normalizing constant calculation of Strauss Process

Suppose that I have the following Strauss Process up to a proportionality constant $$p(\mu_{1}, \mu_{2},..., \mu_{K},K)\propto \xi^{K}\prod_{i=1}^{K} I(\mu_{i}\in R) *a^{\sum_{i,j}|\mu_{i}-\mu_{j}|<...
Fiodor1234's user avatar
  • 2,264
0 votes
1 answer
30 views

Can I transform my output variable in an imbalanced dataset?

I have a dataset that has an output variable that is quite right-skewed and imbalanced. I want to use a neural network as a regressor to predict the output variable. Visually, it looks like there may ...
Omnitragedy's user avatar
0 votes
1 answer
217 views

Image pre-processing for Variational Autoencoder

Setting I am training a Variational Autoencoder (VAE) on the CIFAR10 dataset, which has RGB colors. The VAE uses convolution and transposed convolution layers as well as linear layers to encoder and ...
joba2ca's user avatar
  • 143
1 vote
2 answers
55 views

Should I transform my positively skewed predictor in hierarchical regression?

I'm doing a hierarchical regression trying to understand how intelligence (first predictor) and personality traits (second predictor) influence general knowledge (dependent variable). The problem is ...
sticker's user avatar
  • 11
0 votes
1 answer
33 views

Bias in the normalisation of gene expression due to low counts

I hope it is okay to ask a question because I am one of the silent user or this forum. My background is in biology and I am not so confident in my skills in stats The goal of the analysis is to ...
black_cleaver's user avatar
0 votes
1 answer
53 views

Normalization of Time Series for GARCH

It is generally observed that financial times series is not normally distributed. So I want to clear the following doubts: Is it necessary to normalize time series data especially before proceeding ...
kavita d's user avatar
0 votes
0 answers
19 views

Can this feature be used as input to cnn based on the q-q plot?

I have normalised my input features and target variable using the quantile transform by sklearn. The Q-Q plot of one of the feature after normalisation is given as below. Is this feature normalised? ...
Sara's user avatar
  • 1
0 votes
0 answers
103 views

Standard error of mean of normalized scaled data

I had to normalize my data using this equation : $m ↦ (m - r_{min}) / (r_{max} - r_{min}) * (t_{max} - t_{min}) + t_{min}$ as seen here: scale a number between a range If I calculate the mean of ...
Jean Jacques 's user avatar
0 votes
0 answers
26 views

Understanding PCA plot built on data normalized by two normalization methods

I tried two different normalization methods, generated the PCA plot above on the combined data, and colored the samples by the normalization type. Both normalization methods should give similar ...
Rohit Farmer's user avatar
1 vote
0 answers
21 views

Appropriate tests and normalization approaches for highly non-normal data

I'm trying to ascertain whether a particular type of article, $T$, is associated with higher engagement scores in academic journals, and how substantial the effect size might be. I have raw data of ...
DRG's user avatar
  • 323
0 votes
0 answers
8 views

Normalised Differences

I have a dataset where my main variable is smoking. I have three definitions of smoking based on different questions and there are overlaps. By the first definition I have 100 people who are smokers, ...
Manisha Kapoor's user avatar
0 votes
0 answers
20 views

Normalizing data when retraining on the train and validation set

It is often good practice to normalize training data for numerical stability and faster convergence. When using a train-validation-test split, it is recommended to calculate normalization parameters (...
Vityou's user avatar
  • 11
1 vote
1 answer
180 views

Normalizing a data set by using z-scores from a related data set

I am trying to come up with a normalization for a set of data. My company puts our products through a series of testers (people) to score our products on key metrics. I am able to access all records ...
Citanaf's user avatar
  • 13
0 votes
0 answers
54 views

How to normalize slope of historical financial data?

Im trying to find a method to normalize the slope value for the historical net profits of many companies. But the problem arises when certain companies are clocking profits in thousands, and some in ...
maksumit's user avatar
0 votes
0 answers
4 views

How do I minimize the difference between development and production data after normalization relative to measures of central tendency?

I think that normalization of data is performed assuming that the samples you train on are "representative" of real-world or "production" data. This means I have to constantly ...
Joachim Rives's user avatar
1 vote
1 answer
414 views

Do I calculate the loss of a regression on the normalized or denormalized data?

I worked on implementing a simple MLP network, which should guess a numerical value based on different values. Basically it's a regression task. This was a few months ago. Everything worked fine, the ...
SickerDude43's user avatar

1
2 3 4 5
25