# Tagged Questions

Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

49k views

### How to summarize data by group in R?

I have R data frame like this: ...
19k views

### How should I transform non-negative data including zeros?

If I have highly skewed positive data I often take logs. But what should I do with highly skewed non-negative data that include zeros? I have seen two transformations used: log(x+1) which has the ...
45k views

### In linear regression, when is it appropriate to use the log of an independent variable instead of the actual values?

Am I looking for a better behaved distribution for the independent variable in question, or to reduce the effect of outliers, or something else?
7k views

### Does it ever make sense to treat categorical data as continuous?

In answering this question on discrete and continuous data I glibly asserted that it rarely makes sense to treat categorical data as continuous. On the face of it that seems self-evident, but ...
2k views

### How small a quantity should be added to x to avoid taking the log of zero?

I have analysed my data as they are. Now I want to look at my analyses after taking the log of all variables. Many variables contain many zeros. Therefore I add a small quantity to avoid taking the ...
1k views

### Why is the square root transformation recommended for count data?

It is often recommended to take the square root when you have count data. (For some examples on CV, see @HarveyMotulsky's answer here, or @whuber's answer here.) On the other hand, when fitting a ...
3k views

### When are Log scales appropriate?

I've read that using log scales when charting/graphing is appropriate in certain circumstances, like the y-axis in a time series chart. However, I've not been able to find a definitive explanation as ...
8k views

### How to change data between wide and long formats in R?

You can have data in wide format or in long format. This is quite an important thing, as the useable methods are different, depending on the format. I know you have to work with ...
815 views

### Choosing seasonal decomposition method

Seasonal adjustment is a crucial step preprocessing the data for further research. Researcher however has a number of options for trend-cycle-seasonal decomposition. The most common (judging by the ...
3k views

### Why is gender typically coded 0/1 rather than 1/2, for example?

I understand the logic of coding for data analysis. My question below is on the use of a specific code. Is there a reason why gender is often coded as 0 for female and 1 for male? Why is this ...
4k views

### Interpretation of log transformed predictor

I'm wondering if it makes a difference in interpretation whether only the dependent, both the dependent and independent, or only the independent variables are log transformed. In the case of ...
4k views

### Logistic Regression: Transforming Variables

When transforming variables, do you have to use all of the same transformation? For example, can I pick and choose differently transformed variables, as in: Let, $x_1,x_2,x_3$ be age, length of ...
317 views

### What to do when some time points have heavily skewed responses and some do not in a repeated measures study?

Typically, when one encounters continuous but skewed outcome measures in a longitudinal design (say, with one between-subjects effect) the common approach is to transform the outcome to normality. If ...
5k views

### Transforming proportion data: when arcsin square root is not enough

Is there a (stronger?) alternative to the arcsin square root transformation for percentage/proportion data? In the data set I'm working on at the moment, marked heteroscedasticity remains after I ...
12k views

### When (and why) to take the log of a distribution (of numbers)?

Say I have some historical data e.g., past stock prices, airline ticket price fluctuations, past financial data of the company... Now someone (or some formula) comes along and says "let's take/use ...
3k views

### “Normalizing” variables for SVD / PCA

Suppose we have $N$ measurable variables, $(a_1, a_2, \ldots, a_N)$, we do a number $M > N$ of measurements, and then wish to perform singular value decomposition on the results to find the axes of ...
1k views

### How to summarize categorical data?

I've been struggling with the following problem with hopefully is an easy one for statisticians (I'm a programmer with some exposure to statistics). I need to summarize the responses to a survey (for ...
5k views

### When to log transform a time series before fitting an ARIMA model

I have previously used forecast pro to forecast univariate time series, but am switching my workflow over to R. The forecast package for R contains a lot of useful functions, but one thing it doesn't ...
263 views

### Is visualization sufficient rationale for transforming data?

Problem I would like to plot the variance explained by each of 30 parameters, for example as a barplot with a different bar for each parameter, and variance on the y axis: However, the variances ...
1k views

### Are these formulas for transforming P, LSD, MSD, HSD, CI, to SE as an exact or inflated/conservative estimate of $\hat{\sigma}$ correct?

Background I am conducting a meta-analysis that includes previously published data. Often, differences between treatments are reported with P-values, least significant differences (LSD), and other ...
2k views

### Box-Cox like transformation for independent variables?

Is there a Box-Cox like transformation for independent variables? That is, a transformation that optimizes the $x$ variable so that the y~f(x) will make a more ...
2k views

### SVD dimensionality reduction for time series of different length

I am using Singular Value Decomposition as a dimensionality reduction technique. Given N vectors of dimension D, the idea is to ...
162 views

587 views

### What are the assumptions of negative binomial regression?

I'm working with a large data set (confidential, so I can't share too much), and came to the conclusion a negative binomial regression would be necessary. I've never done a glm regression before, and ...
645 views

### Transformation to increase kurtosis and skewness of normal r.v

I'm working on an algorithm that relies on the fact that observations $Y$s are normally distributed, and I would like to test the robustness of the algorithm to this assumption empirically. To do ...
2k views

### A transform to change skew without affecting kurtosis?

I am curious if there is a transform which alters the skew of a random variable without affecting the kurtosis. This would be analogous to how an affine transform of a RV affects the mean and ...
737 views

### Normal distribution and monotonic transformations

I've heard that a lot of quantities that occur in nature are normally distributed. This is typically justified using the central limit theorem, which says that when you average a large number of iid ...
2k views

### Back-transformation of regression coefficients

I'm doing a linear regression with a transformed dependent variable. The following transformation was done so that the assumption of normality of residuals would hold. The untransformed dependant ...
2k views

### Express answers in terms of original units, in Box-Cox transformed data

For some measurements, the results of an analysis are appropriately presented on the transformed scale. In most of the cases, however, it's desirable to present the results on the original scale of ...
1k views

### Transforming variables for multiple regression in R

I am trying to perform a multiple regression in R. However, my dependent variable has the following plot: Here is a scatterplot matrix with all my variables ...
4k views

### How to do ANOVA on data which is still not normal after transformations?

I'm looking at the effect defeat and entrapment inducing conditions have on subjective ratings of defeat and entrapment at three different time points (among other things). However the subjective ...
293 views

### Analysis with complex data, anything different?

Say for example you are doing a linear model, but the data $y$ is complex. $y = x \beta + \epsilon$ My data set is complex, as in all the numbers in $y$ are of the form $(a + bi)$. Is there ...
1k views

### What is the effect of dichotomising variables?

When dichotomising variables, what information is lost in the process? How does a dichotomisation help in the analyses?
3k views

### Why use logged variables?

Probably, this is a very basic question but I don't seem to be able to find a solid answer for it. I hope here, I can. I'm currently reading papers as a preparation for my own master's thesis. ...
156 views

### How to fairly determine winners for a regional science fair?

I need help figuring out the correct way to calculate winners at our science fair. I don't want my ignorance of statistics & math to get in the way of a kid's chances of winning. (lots of ...
197 views

### How to combine the forecasts when the response variable in forecasting models was different?

Introduction In forecasts combination one of the popular solutions is based on the application of some information criterion. Taking for example Akaike criterion $AIC_j$ estimated for the model $j$, ...
5k views

### Column-wise matrix normalization in R [closed]

I would like to perform column-wise normalization of a matrix in R. Given a matrix m, I want to normalize each column by dividing each element by the sum of the ...
199 views

### How to find a suitable association of color with data value in a visualization?

I'm working on a software project that involves creating a visualizer for flood simulations. As part of this project, I've created a water gradient that shows water depth at particular points. To set ...
5k views

### What could be the reason for using square root transformation on data?

Is there any reason of what I can think of, to transform the data with a square root? I mean what I always observe is that the R^2 increases. But this is probably just because of centering the data! ...
294 views

### How to verify a distribution is normalized?

I have a question in which it asks to verify whether if the Uniform distribution (${\rm Uniform}(a,b)$) is normalized. For one, what does it mean for any distribution to be normalized? And two, ...
235 views

### Finding the distribution of a statistic

Studying for a test. Couldn't answer this one. Let $X_{1,i},X_{2,i},X_{3,i}, i=1,\ldots,n$ be iid $\mathcal{N}(0,1)$ random variables. Define \$W_i = (X_{1,i} + X_{2,i}X_{3,i})/\sqrt{1 + ...
179 views

### Does using box-cox transformation on individual data sets prevent these data from being comparable?

I have been using the box-cox transformation to normalise data for input to an Ecological Niche Factor Analysis software, as recommended by the software creators. However, it has occurred to me that ...
575 views

### Multiple imputation questions for multiple regression in SPSS

I am currently running a multiple regression model using imputed data and have a few questions. Background: Using SPSS 18. My data appears to be MAR. Listwise deletion of cases leaves me with only ...
455 views

### Choosing c such that log(x + c) would remove skew from the population

I have data for which I would like to take the log transformation before doing OLS. The data include zeros. Thus, I want to do a log(x + c). I know a traditional c to choose is 1. I am wondering ...
502 views

### How to model this odd-shaped distribution (almost a reverse-J)

My dependent variable shown below doesn't fit any stock distribution that I know of. Linear regression produces somewhat non-normal, right-skewed residuals that relate to predicted Y in an odd way ...
160 views

### Extract data points from moving average?

Is it possible to extract data points from moving average data? In other words, if a set of data only has simple moving averages of the previous 30 points, is it possible to extract the original data ...
2k views

### How to interpret regression coefficients when response was transformed by the 4th root?

I'm using fourth root (1/4) power transformation on my response variable, as a result of heteroscedasticity. But now I'm not sure how to interpret my regression ...