Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [normalization]

Usually "normalization" means re-expressing data to make values lie within a specified range.

0
votes
1answer
23 views

Scaling different data sets such that transformed values will be between (0,1)? [on hold]

I am creating a model to calculate the weighted average score where parameter values are coming from different datasets. DATASET 1 RANGE: [-1,1] DATASET 2 RANGE: [0,100] DATASET 3 RANGE: (-...
0
votes
0answers
16 views

Normalizing output of Viterbi algorithm

Viterbi algorithm can be used to solve problems in belief networks of the following kind: $$argmax_{x_{1:t}}P(x_{1:t}| e_{1:t})$$ where $e_{1:t} \in E^t$ are evidence variables and $x_{1:t} \in S^t$ ...
-1
votes
1answer
9 views

clustering analysis\ data is in right skewed

what normalization technique(mean normalization/min max/zscore) is prefered to apply k-means clustering scope of data: right skewed variables are different scales & magnitude(days,counts) ...
0
votes
0answers
9 views

Confused about z-score image normalization output

I am trying to normalize my input data for a convolutional network, I applied the z-score normalization technique to my image dataset as follows: Formula: (image - mean(image)) / std(image) ...
0
votes
0answers
15 views

Find normalization constant from factorized density

Please consider the following Bayes Network: We can express density $p(\mathbf{x}_1 | \mathbf{x}_0, \mathbf{y}_1)$ in terms of measurement and motion models by ignoring normalization constants as: $...
2
votes
1answer
35 views

Alternatives to Pre-Scaling Predictors in Lasso/Ridge Regression?

In lasso/ridge regression it's often recommended to scale predictors $X$ before estimation so that the coefficient estimates $\hat{\beta}$ will be invariant to the scale of predictors $X$. Q: Is ...
0
votes
0answers
14 views

In classification problem,do we need to normalize features before fitting a specific model

Given a binary classification problem,do we need to normalize features under methods below? logistic regression KNN (as I know, we should normalize features usually using Min-Max rule) SVM ...
0
votes
0answers
19 views

Normalization in SVM classifier

I am trying to normalize my features for a classification model with 3 class outputs. There are two kinds of features. First is medical test results and second is patient information such as age. The ...
0
votes
2answers
29 views

Should feature normalization be done for training , test set and target variables together? [duplicate]

Consider a matrix X where the columns are the attributes and each row is an example. There is no order of occurrence of the examples. The target variables are ...
0
votes
0answers
25 views

Two way ANOVA with percent change

I have two groups of animals: "active cycle" and "inactive cycle". Within each group, there are two groups: "control" and "treated". I measured the levels of a specific receptor for each animal in ...
0
votes
1answer
17 views

why data normalization is important for models when parameters can manage the feature weight/importance

When we study about normalization, various facts are given to explain the necessity. The most important of all is that: Normalized column if in higher range than others can have more impact on ...
1
vote
1answer
18 views

What is the specific normalization chi2 in seqdist?

In the documentation for the seqdist() function it is noted that there is "...a specific normalization for"CHI2" and "EUCLID". See the Details section." (p.60). But in the details section there is no ...
0
votes
1answer
15 views

How normalize a list of time series features with pyspark?

My goal is to normalize a list of time series to perform a kmeans clustering My dataset is a dataframe with hashtags as entries and column containing time serie features like: ...
1
vote
0answers
17 views

which normalization method is appropriate for regression problem?

I have a regression problem. I have 3 features: X1 is in the range (0.7 , 1.3) X2 is in the range (0.4 , 0.5) X3 is in the range (4.5 , 5.5) and Y is in the range (115 , 719) I started trying ...
0
votes
0answers
13 views

Why should we mean normalize data for low rank matrix factorization?

So I have just studied low-rank matrix factorization lecture by Professor Andrew Ng on Coursera. As an implementation detail, he mentions that we must mean normalize our data. But what are the real ...
0
votes
0answers
43 views

Normalizing after standard scaling

I am a bit confused about this post: https://machinelearningmastery.com/machine-learning-data-transforms-for-time-series-forecasting/#comment-483967 Dr. Brownlee advises on normalizing the dataset ...
0
votes
0answers
8 views

Feature scaling/normalization follwing a sample level normalization

In micro-array or many similar platforms per sample normalization (z-score) is a common practice to minimize the impact of outlier values. Do we need a feature scaling after this ? In Details : I ...
1
vote
1answer
47 views

Density Estimation and Data Normalization

Is there any problem to first normalize data (for example, min-max one) then use kernel density estimation to get pdf of each sample? Thanks.
3
votes
1answer
33 views

Should I subtract lower bound from Gamma distributed data before estimating distribution parameters?

I have some real world data that reflects waiting time in a system. As it's about waiting times I assume it's Gamma distributed and visual check (histogram overlaid by a fitted Gamma PDF) shows no ...
1
vote
0answers
23 views

Scaling Variables in PCA, yet all on the same scale

I know this topic of scaling and normalizing variables for PCA has been posted on a lot, 1, 2, 3. However, I am performing PCA on coordinate data that is measured all on the same scale, i.e. (x,y) ...
0
votes
0answers
16 views

Efficiently normalize word embeddings

I'm using glove word embedding and would like to [-1,1] normalize it using python. The data is in the format of a dict with the word as key and a ...
0
votes
0answers
5 views

Speaker normalization of features before model training

I am building a model using a supervised machine learning based on features I extract from speech signals. The features include MFCC, auto correlation and energy derivatives. According to this paper,...
0
votes
1answer
39 views

Likelihood - not a pdf and not normalized?

I am reading the book "Patter recognition" by Cristopher Bishop. At Chater 2.3.6 "Bayesian inference for the Gaussian", there is written The likelihood function, that is, the probability of the ...
2
votes
1answer
44 views

In Convolutional networks, how to do input data normalization? is it necessary?

I'm wondering about data normalization in CNN, how can we do it for the input images?, what can it add to the model's performance? and what are the main pre-processing techniques before doing the ...
0
votes
0answers
3 views

Normalise exponential numbers between 0 and 1 [migrated]

I'm creating a fractal visualisation. I want the colour per point to be based off the iteration final value, $f(z_{n})$, instead of the traditional: number of iterations before reaching a cut-off (...
0
votes
1answer
18 views

How to standardize data with low variance?

I have quarterly data of federal fund rate (test set), e.g.: ...
0
votes
0answers
33 views

scale with z-transform with the outliers in the data

I have 2 different columns X1, and X2. As you can see scale of X1 is larger than X2. These 2 columns represent the predicted value of performance of my two products. However the range or scale of X1 ...
1
vote
1answer
48 views

Does data normalization and transformation change the Pearson's correlation?

As we know that Pearson's correlation measures the linearity between two variables, I am wondering when applying normalization and transformation on the original dataset, does the normalization and ...
0
votes
1answer
31 views

Am I Log Normalizing correctly?

I am sorry if this is a stupid question, but I have searched the other questions about log normalization found other sources on it all of which assume a level of understanding that I don't have. I ...
1
vote
1answer
28 views

Preprocessing gene expression dataset

Given a gene expression dataset with 99 samples and 10000 features, it is required to find clusters of samples in the dataset. Now taking the features and finding their means and subtracting those ...
0
votes
1answer
36 views

Standardization of Data

I have a dataset which consists of Sales for Product1 and Product2. It also tells if the <...
0
votes
0answers
24 views

Self-study: Compute mean and std dev

Suppose that for a TV model it is known that 60% lasts more than 3 years, and 70% lasts less than 6 years. Assuming TV life follows a normal distribution. What is the mean and std dev? These are the ...
0
votes
0answers
43 views

Log transform with 'zero' values [duplicate]

I am doing some explorative work on two large datasets. One from 2001 and one from 2018. The dataset consists of measured soil-parameters and it contains lots of zero's. From the transformations ...
0
votes
0answers
15 views

Normalization of data with different ratings

I have a set of data provided by 6 different machines. I want to know which is the most efficient, but each machine has a different power rating, ie. One is 5 kW, while the other is 3kW, and 1.5 kW, ...
3
votes
1answer
79 views

Which machine learning algorithms get affected by feature scaling?

Which of the following machine learning algorithms will be affected if we apply feature scaling? Naïve-Bayes k-Nearest Neighbor (KNN) Support Vector Machine (SVM) Decision Trees Neural Network (...
0
votes
0answers
7 views

Which normalization package can I use for different types of factors (length, weigth, etc

I'm doing a study of toxicology (heavy metals) in fishes. I want to do a linear regression analisis but first i must normalize my database. But i have got different columns with different units: ...
1
vote
1answer
46 views

proper way to scale and plot data points on top of each other

I hope this is the right place that I am posting this question. If not please feel free to comment so that I find the right place. I have 4 sets of points that represent points on hexagons. My data ...
0
votes
1answer
24 views

How to comparte two metrics, from profile assessment, with different distributions

I have four metrics from an assessment test: Energy; Emotional Volatility; Planning Skills; and Creativity. Each metric has a scale that goes from 1 to 5 (ex: 4.5....1.3) How can i compare one ...
0
votes
0answers
20 views

Normalised Root Mean square error

I have $10$ people in a group and they undergone a surgery. I have the root mean square of each subject before and after the surgery. ...
0
votes
0answers
14 views

I have question about normalization of human pose estimation

I have a question about normalization of object joint detection here is the description about the normalization method Further, since the joint coordinates are in absolute image coordinates, it ...
1
vote
1answer
36 views

Which constant to add when applying 'Box-Cox transformation' to negative values?

Questions How big constant should we add to negative values when applying the 'Box-Cox transformation'? The data that I am handling is 'daily return of stocks' Shouldn't we subtract some amount after ...
0
votes
0answers
15 views

CNN Feature Normalization: Looking into the Future?

It occurs to me as I'm normalizing the features for my CNN that I am inadvertently taking the "future" into account by normalizing using the min and max of the entire time period. In other words, at ...
0
votes
0answers
27 views

Histogram equalization

I'm trying to equalize the following histogram: I tried histogram equalization, but that didn't seem to work the way I wanted. I got something likes this: Which is equally distributed everywhere. ...
1
vote
1answer
64 views

How to use Cullen and Frey graphs for downstream statistical analysis? [closed]

How to use Cullen and Frey graphs for downstream statistical analysis? I do not see any difference in normalized and raw data Cullen and Frey plots! Raw data ...
0
votes
0answers
8 views

Should I normalise my data for PCA, Sammon and SOM mapping? [duplicate]

I do not think I need to log-transform my data as the distribution of the components are not skewed. However, the scale of the components are different (i.e. Age, resting blood pressure, maximum heart ...
0
votes
0answers
6 views
0
votes
1answer
16 views

How to interpret normalized difference score?

Lets say you have 2 variables, reaction time (RT) on an easy task and reaction time on a hard task. I've seen papers calculate a normalized difference score using the following: $$ Z = \frac{RT(hard) ...
0
votes
1answer
77 views

K-means clustering scaling

I have a data set of 70 stores with a sales column (ranging from 50M to 70M) and 39 other features, like age group, income categories etc. I need to find the clusters based off of these metrics. A ...
3
votes
2answers
298 views

Simulating rnorm() using runif()

I am trying to 'simulate' rnorm() using only runif(). I don't know if I should do: sqrt(-2*log(U1))*cos(U2) or ...
0
votes
0answers
15 views

Data transform for time series with value regime

I have a time series data which looks as following: I need to understand the relationship of this data vs. daily temperature. The spikes in data for Jan 2009 and Sept 2009 are caused by certain ...