# Questions tagged [data-transformation]

Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

321 questions
Filter by
Sorted by
Tagged with
5k views

### PDF of cosine of a uniform random variable

There is a formula for the density of the cosine of random variable that's a uniform on $(-\pi,\pi)$ as discussed in this page: $f_{Y}(y) = \dfrac{1}{\pi \sin(\cos^{-1}y)}, y \in\ [-1,1]$ Can anyone ...
2k views

### Should quantitative predictors be transformed to be normally distributed?

I am always struggling with normality testing for quantitative predictors (no factors) and transforming them to normality. If I am running a GLMM and my predictors are really non-normal, should I ...
2k views

### Appropriate data transformation

I have two dependent variables y1 and y2 with highly skewed distributions. In order to do ANOVA, I was trying to transform the ...
301 views

### Natural log approximation

I've got an equation that contains $$x^p - 1$$ $x$ is any positive number (such as 2) and $p$ is a small positive number close to 0 (such as 0.001). For some reason (that I may have known in High ...
51k views

### One-hot vs dummy encoding in Scikit-learn

There are two different ways to encoding categorical variables. Say, one categorical variable has n values. One-hot encoding converts it into n variables, while dummy encoding converts it into n-1 ...
32k views

### Is a log transformation a valid technique for t-testing non-normal data?

In reviewing a paper, the authors state, "Continuous outcome variables exhibiting a skewed distribution were transformed, using the natural logarithms, before t tests were conducted to satisfy the ...
487 views

### Intuition behind Box-Cox transform

For features that are heavily skewed, the Transformation technique is useful to stabilize variance, make the data more normal distribution-like, improve the validity of measures of association. I am ...
33k views

### ANOVA on binomial data

I am analyzing an experimental data set. The data consists of a paired vector of treatment type and a binomial outcome: ...
3k views

### Are these formulas for transforming P, LSD, MSD, HSD, CI, to SE as an exact or inflated/conservative estimate of $\hat{\sigma}$ correct?

Background I am conducting a meta-analysis that includes previously published data. Often, differences between treatments are reported with P-values, least significant differences (LSD), and other ...
11k views

### Standardizing features when using LDA as a pre-processing step

If a multi-class Linear Discriminant Analysis (or I also read Multiple Discriminant Analysis sometimes) is used for dimensionality reduction (or transformation after dimensionality reduction via PCA), ...
5k views

### Help me fit this non-linear multiple regression that has defied all previous efforts

EDIT: Since making this post, I have followed up with an additional post here. Summary of the text below: I am working on a model and have tried linear regression, Box Cox transformations and GAM but ...
5k views

### How is the Box-Cox transformation valid?

The Box-Cox transformation transforms our data into a normal distribution. How is that even a proper technique? What if our data didn't come from a normal distribution? How could someone just blindly ...
20k views

### Box Cox Transforms for regression

I'm trying to fit a linear model on some data with just one predictor (say (x,y)). The data is such that for small values of x, the y values give a tight fit to a straight line, however as x values ...
7k views

### Back-transformation and interpretation of $\log(X+1)$ estimates in multiple linear regression

I have performed multiple linear regression analyses with different combinations of transformed and untransformed variables--both explanatory (independent) and response (dependent) variables. All ...
475 views

684k views

### How to summarize data by group in R? [closed]

I have R data frame like this: ...
22k views

### What are the assumptions of negative binomial regression?

I'm working with a large data set (confidential, so I can't share too much), and came to the conclusion a negative binomial regression would be necessary. I've never done a glm regression before, and ...
2k views

I'm looking for an advanced linear regression case study illustrating the steps required to model complex, multiple non-linear relationships using GLM or OLS. It is surprisingly difficult to find ...
20k views

### Why log-transforming the data before performing principal component analysis?

Im following a tutorial here: http://www.r-bloggers.com/computing-and-visualizing-pca-in-r/ to gain a better understanding of PCA. The tutorial uses the Iris dataset and applies a log transform prior ...
1k views

### Pitfalls to avoid when transforming data?

I achieved a strong linear relationship between my $X$ and $Y$ variable after doubly transforming the response. The model was $Y\sim X$ but I transformed it to $\sqrt{\frac{Y}{X}}\sim \sqrt{X}$ ...