# Questions tagged [data-transformation]

Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

321 questions
Filter by
Sorted by
Tagged with
5k views

### PDF of cosine of a uniform random variable

There is a formula for the density of the cosine of random variable that's a uniform on $(-\pi,\pi)$ as discussed in this page: $f_{Y}(y) = \dfrac{1}{\pi \sin(\cos^{-1}y)}, y \in\ [-1,1]$ Can anyone ...
2k views

### Should quantitative predictors be transformed to be normally distributed?

I am always struggling with normality testing for quantitative predictors (no factors) and transforming them to normality. If I am running a GLMM and my predictors are really non-normal, should I ...
2k views

### Appropriate data transformation

I have two dependent variables y1 and y2 with highly skewed distributions. In order to do ANOVA, I was trying to transform the ...
301 views

### Natural log approximation

I've got an equation that contains $$x^p - 1$$ $x$ is any positive number (such as 2) and $p$ is a small positive number close to 0 (such as 0.001). For some reason (that I may have known in High ...
51k views

### One-hot vs dummy encoding in Scikit-learn

There are two different ways to encoding categorical variables. Say, one categorical variable has n values. One-hot encoding converts it into n variables, while dummy encoding converts it into n-1 ...
32k views

### Is a log transformation a valid technique for t-testing non-normal data?

In reviewing a paper, the authors state, "Continuous outcome variables exhibiting a skewed distribution were transformed, using the natural logarithms, before t tests were conducted to satisfy the ...
487 views

### Intuition behind Box-Cox transform

For features that are heavily skewed, the Transformation technique is useful to stabilize variance, make the data more normal distribution-like, improve the validity of measures of association. I am ...
33k views

### ANOVA on binomial data

I am analyzing an experimental data set. The data consists of a paired vector of treatment type and a binomial outcome: ...
3k views

### Are these formulas for transforming P, LSD, MSD, HSD, CI, to SE as an exact or inflated/conservative estimate of $\hat{\sigma}$ correct?

Background I am conducting a meta-analysis that includes previously published data. Often, differences between treatments are reported with P-values, least significant differences (LSD), and other ...
11k views

### Standardizing features when using LDA as a pre-processing step

If a multi-class Linear Discriminant Analysis (or I also read Multiple Discriminant Analysis sometimes) is used for dimensionality reduction (or transformation after dimensionality reduction via PCA), ...
5k views

### Help me fit this non-linear multiple regression that has defied all previous efforts

EDIT: Since making this post, I have followed up with an additional post here. Summary of the text below: I am working on a model and have tried linear regression, Box Cox transformations and GAM but ...
5k views

### How is the Box-Cox transformation valid?

The Box-Cox transformation transforms our data into a normal distribution. How is that even a proper technique? What if our data didn't come from a normal distribution? How could someone just blindly ...
20k views

### Box Cox Transforms for regression

I'm trying to fit a linear model on some data with just one predictor (say (x,y)). The data is such that for small values of x, the y values give a tight fit to a straight line, however as x values ...
7k views

### Back-transformation and interpretation of $\log(X+1)$ estimates in multiple linear regression

I have performed multiple linear regression analyses with different combinations of transformed and untransformed variables--both explanatory (independent) and response (dependent) variables. All ...
475 views

684k views

### How to summarize data by group in R? [closed]

I have R data frame like this: ...
22k views

### What are the assumptions of negative binomial regression?

I'm working with a large data set (confidential, so I can't share too much), and came to the conclusion a negative binomial regression would be necessary. I've never done a glm regression before, and ...
2k views

I'm looking for an advanced linear regression case study illustrating the steps required to model complex, multiple non-linear relationships using GLM or OLS. It is surprisingly difficult to find ...
20k views

### Why log-transforming the data before performing principal component analysis?

Im following a tutorial here: http://www.r-bloggers.com/computing-and-visualizing-pca-in-r/ to gain a better understanding of PCA. The tutorial uses the Iris dataset and applies a log transform prior ...
1k views

### Pitfalls to avoid when transforming data?

I achieved a strong linear relationship between my $X$ and $Y$ variable after doubly transforming the response. The model was $Y\sim X$ but I transformed it to $\sqrt{\frac{Y}{X}}\sim \sqrt{X}$ ...
4k views

### Why aren't power or log transformations taught much in machine learning?

Machine learning (ML) uses linear and logistic regression techniques heavily. It also relies on feature engineering techniques (feature transform, ...