# Questions tagged [data-transformation]

Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

2,305 questions
Filter by
Sorted by
Tagged with
5 views

### How does using log transformation for a confidence interval of the survival stabilizes variance?

I only found that it eliminates the estimator of survival from the variance formula, but could anyone show some references or write a few formulas to show how exactly the Greenwood becomes more stable ...
29 views

### What exactly does the Box-Cox transformation do to a time series?

If I were to try and rephrase the argument in the original Box-Cox paper in my own words, I would say something like the following: given a model $$y = x \beta ,$$ if the residuals do not appear to ...
• 314
1 vote
25 views

### Derivation of Box-Cox and Yeo-Johnson Log-Likelihood Functions

The scipy documention lists expressions for the Log-likelihood functions for the Box-Cox and Yeo-Johnson transformations here and here. I'm looking for a source ...
• 314
13 views

### STATA: convert from wide to long format [closed]

I tried to convert according to the letter "y", thats why i added the letter to the variable names. The structure of the data is pictured below. Im trying to group the "kommune" ...
1 vote
28 views
+50

### How to learn steep functions using neural network?

I am trying to use a neural network to learn the below function. In total, I have 25 features and 19 outputs. The above image shows the distribution of two features with respect to one of the outputs....
• 165
7 views

### Which is the condition for using log transform for data before taking cointegration test?

I saw some discussions on whether to use log transformation on data before taking the cointegration test, many people recommend taking log transformation before the test. I want to ask if there are ...
5 views

### Log input data multi target regression [duplicate]

I'm building a multi target regression model with random forest regressor, in python. My database has 20 variables, from which 13 are x values and 7 are y values (targets). I've treated 3 of these 13 ...
24 views

### Reverting differencing for Time Series VAR model component wise?

I have data with seasonality so I used seasonal and then first order differencing to make the series stationary in order to fit VAR model. After the model is fitted on the differenced level I can ...
35 views

### Derive the pdf/cdf of a variable given as a formula of two random variables [closed]

Let's assume I have two random variables X and Y with x>0 and y>0, respectively. Let's also assume that their marginals are known as well as the joint cdf is known as the product of the two ...
• 21
1 vote
13 views

### Is there a difference between kernel PCA with a non-linear kernel vs PCA with a non-linear change of variables?

I see that kernel PCA with a linear kernel is the same as PCA. On Wikipedia's introduction of the kernel to PCA they suggest that there exists a non-trivial arbitrary choice of map $\Phi$ that is ...
• 4,560
53 views

### modelling proportion data - transforming 0 and 1s to fit beta regression

I have proportion data which includes 0s and 1s (the ri column below, calculated from the ud/days_lib)). I have found a few threads related to modelling this issue. A few of which (e.g., this thread) ...
11 views

### Instrumental Variable Transformation Question on Fixed Effects model

My question is regarding something my professor did, but I didn't understand the purpose of it. I have panel data for 1052 municipalities from the year 2003 to 2019. I have a variable that tells me ...
• 1
1 vote
64 views

### Does log-transforming a slightly-skewed y variable make any sense?

I'm separately analyzing 5 subsets of my data and running multiple linear regressions with the same outcome variable in each. (I am also running a single regression with all the subsets and using the ...
25 views

### Jacobian of function returning $m$ evenly-spaced order statistics of an $n$-dimensional vector

Let $y\in\mathbb{R}^n$, and let $f:\mathbb{R}^n\to\mathbb{R}^m$ be the transformation that outputs $m$ evenly-spaced order statistics (including the extremes) of $y$. What is the Jacobian of this ...
• 1,560
7 views

### Johnston transformation of censored data for tobit regression, to estimate the proportion pitted beyond the specification depth

I have a sample of 35 components from a population of machines. The depth of pitting on all these components has been measured. The resulting dataset looks to be left censored, a large number of no ...
• 11
19 views

### Multiple comparisons of Box-Cox transformed data

I'm working on a dataset of highly-skewed data that I have transformed using Box-Cox. I have 2 groups (healthy controls and diseased participants) and I need to perform multiple comparisons (to ...
• 1
98 views

### Interpret bell curve, calculate 95th percentile value

I have data showing the distribution of a given metric across a population. The data is broken up into increments, with values for the 90th 97.5th and 99th percentiles. How can I interpret this data ...
• 23
1 vote
23 views

### Comparison of different IRFs from the VAR model (meta-analysis)

In my meta-analytic research, I have collected IRFs from papers where two variables x and y are both in log-levels (these variables were entered into some VAR model in this way). In addition, I also ...
• 11
491 views

### What's the right way to rescaling (min-max normalization)?

Say I have a scale 2-12 and want to transform it into the range between 0-1. I would use the following formula: $$y={\frac {x-{\text{min}}(x)}{{\text{max}}(x)-{\text{min}}(x)}}$$ Sometimes I just saw ...
• 31
14 views

### What is the right order in dealing with outliers, missing values and log transformation?

I am currently working on a project involving banking stock price data. I have around 3000 observations, some columns have a lot of missing values (null value); they can account for 5 to 50% of the ...
313 views

### Log of a log-transformed variable

I have been suggested to use the log of a log-transformed independent variable (i.e., log(log healthcare expenditure)). I am not sure how would this make sense. Is this a standard practice (in the ...
• 61
43 views

### How do you reverse log, square root, and Ln data transformations?

If you have transformed your data, is it true that when reporting results, such as descriptive statistics (e.g. mean, median, range, variance, standard deviation etc.), you need to revert the data ...
22 views

• 12.4k
61 views

### Standardize dataset with high outliers

Is there a better way to standardize a dataset with outliers than to normalized value (z-score) based on the mean and standard deviation? I am using the Excel STANDARDIZE function. I have two datasets ...