Questions tagged [normalization]

Usually "normalization" means re-expressing data to make values lie within a specified range.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

What is the most efficient way of adding values from different scales to create a metric? Normalization v/s Scaling v/s?

I'm working on creating a metric to evaluate medical protocols with different factors. The proportion of weightage of each factor is determined by a survey which we have calculated as below : Factor a ...
0
votes
0answers
14 views

Should one-dimensional data be normalized for K-Means clustering?

Data normalization is important prior to K-Means clustering when there are multiple variables in the clustered data set. Data centering and scaling (for instance using Z-score) can change the relative ...
0
votes
0answers
11 views

Quantify a score (or figure-of-merit) of the code style for source code of different sizes

I want to have a numeric score between 0 and 100 to reflect the quality of code style for the R source code written by my team members based on some pre-defined rules. The rules and checks are taken ...
0
votes
0answers
8 views

Normalization changes the effect of a variable in fixed effect logit

I have constructed a conditional/fixed effect logit model. My data consists of 6k+ groups in which I get two different coefficients for the same variable if I divide that variable to the the size of ...
0
votes
0answers
27 views

Normalize skewed distribution

My data is right-skewed. Log-transform data only shifts it, not changing the distribution shape. Tried to use QuantileTransformer but output seems to be really messy. Any suggestions on how to ...
0
votes
0answers
19 views

How to normalize predicted values for an outcome event?

I work on predictive models for crime forecasting, meaning I try to model the risk for crimes. In the end of my modeling, I have the following values: number of predicted crimes for each state (...
1
vote
1answer
78 views

Why does normalizing image twice work?

I made a 'mistake' while training a neural network, it is a typical image classification problem like this. However the data is much larger and came from Kaggle. In my ...
0
votes
1answer
29 views

What is the l1-normalization of some data?

From this page and in this paper (first paragraph of chapter 2.1) there is the term of "$l_1$-normalization" or absolute normalization of a vector (i.e. some data). The scope is to turn the ...
2
votes
1answer
25 views

Scaling embedding layer's outputs in Tensorflow

I have a neural network that takes categorical and quantitative features as inputs. The quantitative features are scaled in $[0,1]$. I apply an embedding layer to get a continuous representation of ...
0
votes
0answers
18 views

Comparing Mean and standard deviation from a normalized histogram and scipy.stats.lognormal fit

I am trying to fit a lognormal distribution to my experimental Data. The experimental data provides the frequency of occurrence of a certain length. The data is given in the code below. The counts_exp ...
0
votes
0answers
30 views

Is it wrong to use tanh with images normalized in [0,1] range?

I've seen in some repositories, mostly related to GANs (Generative Adversarial Networks) using tanh activation function whilst having input images in the range of <...
0
votes
0answers
16 views

Standardizing Multiple Multivariate Time Series

I have a set of devices that I am using to collect data, each device collects a multivariate time series at the same sampling rate. (around 10 minutes). I have done first-order differencing on the ...
0
votes
0answers
11 views

Data hybrid logarithmic normalization for ANN

is it ok to normalize data by using log normalization before trained the ANN model as following: xn=log(x) yn=log(y) then performing min-max normaliztion as following: xnn=(xn-min)/(max-min) ynn=(yn-...
1
vote
0answers
17 views

Normalizing a Categorical Ordinal Variable

I am trying to normalize my data in which I collected number of time people pick their hair and their stress level which ranges from 1 (no stress) to 5 (high stress) and try to create a graph in order ...
0
votes
1answer
28 views

Matrix row - PCA

I know that before doing PCA we need to normalize and scale the data by feature. I am wondering what it will be happened if the normalization and the scaling operations would be calculated for each ...
0
votes
1answer
12 views

What are some existing techniques for pose estimation angle normalization?

So I am currently building a model which does a certain type of action recognition, which I am implementing as a two-stage, end-to-end system. The first stage is a pose estimation model, and I want to ...
1
vote
0answers
12 views

Data transformations for severely positively skewed data [closed]

I have 2 variables in my dataset that are severely positively skewed. I have tried the most common transformations to no avail, including log, inverse, and Box-Cox. Is there any other type of ...
0
votes
0answers
7 views

Standardization of Google Trends data

Can I standardize Google Trends data that is already normalized? I thought about subtracting the mean and dividing by the deviation, the most popular way of standardization.
1
vote
0answers
22 views

t-SNE and normalization / standardization

I have 7 parameters with different scales. If I understand correctly, before applying t-SNE or other dimension reduction methods I should apply a standardization or normalization method on my ...
0
votes
0answers
20 views

How to normalize data that have features with different ranges and not knowing min/max?

I have a dataset of network flows with several numerical features that range differently (from [0, 6e6] to [0, 1]) and some of ...
0
votes
0answers
23 views

Can different types of feature scaling result in different prediction performance and how to choose one type?

Being new to machine learning and currently making use of a MLP-Classifier from scikit learn to solve a multi-class multi-label classification problem, I was wondering how to decide on a type of ...
0
votes
0answers
39 views

Reduce skewness of normalized weights with outliers

I have a vector of values and I want to extract their weights so that they sum up to 1. I currently use this simple formula for normalization: $ w^i = \frac{ x^i }{ \sum {x} }$ The problem is that ...
0
votes
0answers
10 views

unable to remove skewness from my data

I am trying to remove skewness from my data, since my linear model requires it. e.g. all the columns in my data look like this after plotting a kde: my data contains, 0s, posituve and negative values....
0
votes
1answer
33 views

Standardizing data values [duplicate]

I see an example for standardizing data value here. For better illustration, the table is shown below I did the same thing in excel and as you can see, the results are different. For mean and ...
0
votes
2answers
16 views

Why do we normalize test data on the parameters of the training data?

I just built a toy linear regression model with gradient descent, coding it from scratch. It was doing fine on test data, but it was off on training data. In the end I figured that I was normalizing ...
1
vote
0answers
18 views

The definition of Normalized Standard Scores [closed]

I am studying a past psychology exam paper at university, and I came across this question. ...
0
votes
0answers
51 views

Best suitable type of Intraclass correlation method?

I would like to understand if the below data is suitable to run single_raters_absolute IntraClass Correlation (ICC) or Single_random_raters or Single_fixed_raters. I have executed the below program in ...
0
votes
0answers
43 views

What is the interpretation of the negative value for the Normalized Mean Absolute Error (nMAE) metric?

I am using the normalized mean absolute error metric for evaluating my results. The data I use is in time-series form. Their trend may be increasing or decreasing over time. All the values are ...
0
votes
0answers
7 views

Normalisation of microbiotadata, TMM

About normalization of microbiota data (TMM method). I dont really understand how to perform it. Below is an example. So it normalizes per group. But if I have a "Dataset" (see below) I want ...
2
votes
1answer
23 views

What is the difference between standardizing time series data and non-time series data?

From reading some answers on this site (1, 2, 3 and 4) I found that, on time series data, standardization must be applied separately on the train and test sets to avoid data leakage. So the train data ...
2
votes
2answers
143 views

What is the name for normalization $\leq 1$?

In my current thesis I have two weight components. As I want to join those components, weighted by a percentage, I thought about normalizing(/scaling?) both components respectively by their max value. ...
0
votes
1answer
17 views

Approximate / Standardize value in certain range

I have table with numeric values like ...
0
votes
0answers
9 views

Are there problems with using Ordered Quantile (ORQ) normalization for Multivariate Analysis?

This is intended as a general question, as a Google search for -- "Ordered Quantile" "ORQ" normalization "Multivariate Analysis" -- returns nothing. This option is listed ...
1
vote
1answer
28 views

Why does normalising a conditional probability sufficient to ensure ensure that the resulting quantity is a distribution?

The following is said in David Barber's Bayesian Reasoning and Machine Learning: "The relation between the conditional p(A=a | B=b) and the joint p(A=a, B=b) is just the normalisation constant ...
0
votes
1answer
31 views

Does PCA normalize data with respect to standard deviation?

In my textbook it says the data is assumed to be zero-centered. I am not sure what this means. I read it stand for substracting the mean from the individual values, however I would like to know do you ...
0
votes
0answers
86 views

How to normalize data for LSTM?

For learning purpose I am making simple dataset for LSTM, which can predict next number in sequence. Here is my x value ...
0
votes
0answers
14 views

How to normalize a value from a range to another [duplicate]

I have a set of value with range 0 to 1. I have to change the dataset from 0 to x. Which formula i have to use?
0
votes
0answers
14 views

Regression : Variable transformation necessary? If yes, why?

I am working on a real estate project and have historical rental prices and vacancy data. Interested in exploring the relationship between vacancy rates and change in rental prices. The unit of ...
0
votes
3answers
127 views

How to interpret normalized coefficients in logistic regression?

I trained a logistic regression model with 5 features per sample. Before training, I normalized the range of my features into [0,1] (MinMax scaler). After training, I received the following ...
1
vote
1answer
34 views

Normalising data for a learning experiment on alpine skiing

We are currently performing a large learning experiment on alpine skiing. In this experiment, we test skiers on three different slalom courses. We compute performance by calculating the average of the ...
0
votes
0answers
33 views

How to convert standard normal output to predictions?

I am using a linear model to perform predictions. My loss function is minimizing three seperate objective functions. In order to perform regression, I converted my 3 objective outputs to standard ...
1
vote
1answer
26 views

Normalize sample to match the mean and the standard deviation

There are two samples (sufficiently large and independent). One, size $n_1$ has the mean $m_1$ and the standard deviation $s_1$, the other, size $n_2$ with the mean $m_2$ and the standard deviation $...
0
votes
0answers
20 views

A question about normalization in probit model (binary response model with normal error)

Suppose I have data on $\{Y_i,X_{1i},X_{2i}\}_{i=1}^{N}$ and the data generating process is $Y_i=\mathbf{1}(\beta_1X_{1i}+\beta_2X_{2i}>e_i)$, where $e_i\sim N(0,\sigma^2)$. Usually, we do a ...
0
votes
0answers
12 views

normalization or standardization for outlier?

I am trying to build a machine learning model that uses the age of the student as a feature. But my data has some erroneous value. which technique should I use to ignore those values? Z-score or ...
2
votes
2answers
35 views

Comparing yearly data that has different $n$ values per year?

I have been collecting soil moisture data at my research site for almost 20 years. However, I have not been consistent with the number of times that I have been taken these measurements. The problem ...
0
votes
0answers
18 views

Fitting time series $b$ to time series $a$ stock prices

This is a bit hard to explain by words but I will do my best and then provide an illustration afterwards. In short I want to fit time series $b$ to time series a while retaining the "uniqueness&...
1
vote
1answer
28 views

Can we normalize both continuous and discrete numerical values

I have a sensor dataset with 16 features as numerical values (12 are continuous and 4 are discrete). I am using LSTM model to fit the data and do some classification. As both continuous and discrete ...
1
vote
0answers
52 views

Why clustering in a linear scale using correlation based distance gives better results than clustering in a log2 scale?(PAM clustering)

I have questions regarding cluster analysis. I am trying to cluster data made up of proteins. (23 columns and 1800 rows) I have the data in a log2 scale, some variables range between 2-10 and others ...
2
votes
1answer
62 views

Preprocessing of target data set in Transfer learning approach

So the idea of transfer learning approach is to pre-train a model on source data set and then re-train (or fine-tune) the model on the target data set. But what about preprocessing? If I choose to ...
1
vote
1answer
26 views

Does model need normalized data?

With a very limited time to look at the model's architecture, how does one decide whether or not an arbitrary model need normalized input data? There are tons of ML libraries out there and most of the ...

1
2 3 4 5
21