# Heteroscedasticity still present after Box-Cox transformation

I just started to learn regression and I'm trying to fit a linear regression model to some data with one continuous independent variable x1, one categorical variable x2, and the dependent variable y.

I first used OLS but the residuals violate both normality and homoscedasticity assumptions:

Then I performed Box-Cox transformation to the y variable as it's originally highly left skewed. After the transformation I tried OLS again, and got another set of residuals that look like this:

I went on to use WLS instead of OLS to handle heteroscedasticity but I'm not sure whether it's the right way to do it.

1. How should I interpret the 2 residual plots? (the 2nd resembles a cone shape which suggests heteroscedasticity, but it's different from a regular cone shape and doesn't center on the x=0 axis with a few points on the upper left, I don't understand why)

2. Should I use a different model (like glm) instead of linear regression? I'm asking this because I noticed that the relation between the continuous variable x1 and y looks not so linear after the Box-Cox transformation of y (which is bizarre for me):

1. It's a dataset between turtle hatch rate (y), infection rate (x1) and species (x2). The original distribution of y, and scatterplot for y-x1, x2 are as below (after removing some outliners):

2. For the transformation of y, I did scipy.stats.boxcox(y). The lambda is auto-computed by the function and here it is 3.16. The y comes out like this:

1. It's a dataset between turtle hatch rate (y), infection rate (x1) and species (x2). The original distribution of y, and scatterplot for y-x1, x2 are as below (after removing some outliners):

2. For the transformation of y, I did scipy.stats.boxcox(y), which came out like this, and now the scatterplot is like:

• What are your variables? One is infection rate, but what are the others? Aug 18, 2023 at 11:35
• Also, I would only transform variables for substantive reasons, not statistical ones. You can use a model that doesn't make the assumptions that OLS does, but if you are just starting to learn regression, then it might be better to wait on this problem and do some others. How are you learning? Are you taking a course? Studying a book on your own? (If so, which one?) Aug 18, 2023 at 11:37
• @PeterFlom Hi Peter, the other variable is species which has 2 levels. I'm a new grad student but the school year hasn't started yet so right now I'm doing self study (online courses mainly).
– Vera
Aug 18, 2023 at 12:17
• Can we please see a histogram of your outcome (dependent variable) and be told what transformation you used? Aug 18, 2023 at 12:34
• I am more positive about transformations than Peter, but my problem is simpler: I need to see a scatter plot of your original variables for a view of whether that suggests a straight line relationship, either before or after some modest transformation. I suspect we need more biological insight here. For example, there is a general but not universal cut-off at about 0.9 for the predictor, whatever it is. (Separate points according to the distinct values of the categorical variable.) Aug 18, 2023 at 12:40

Playing with transformations creates great inference uncertainty in the final model, distorting standard errors etc. And for many transformations one does not know the origin to subtract before taking the transformation, i.e., taking logs on the raw data is not always correct even if log is correct. IMHO it is far better to use transformation-invariant methods that are powerful and don't create extra model uncertainties. Consider the use of semiparametric ordinal regression which works perfectly well for both continuous Y and for discrete ordinal Y. Semiparametric methods are virtually as powerful as parametric ones if parametric assumptions hold, and are almost always more powerful when not.

• Hi Frank! Could you explain a bit about what you mean by " taking logs on the raw data is not always correct even if log is correct"?
– Vera
Aug 18, 2023 at 13:47
• There are examples where to satisfy assumptions one needs to take $\log(x + c)$ where $c$ is larger than what you expect, and is larger than just what is needed to ensure positivity of what's being logged. $\log$ is very origin-dependent. Taking logs of ratios usually works without needing to worry about that. Aug 18, 2023 at 18:14
• Note that reservations about needing to choose $c$ in log $(y + c)$ have no bearing on my suggestion of using a log link, even if zeros are observed in the data. Aug 18, 2023 at 18:34

It's a great disappointment not to have access to the data to test these guesses. Wanting to respect the science is as or more important than catering to fetishes or phobias about the form of the data.

I assume

A1. Hatch rate $$H$$ is the proportion of turtle eggs that hatch, so it's bounded by 0 and 1 and to make biological sense the predicted or fitted values should be so bounded.

A2. Infection rate $$I$$ is another proportion. It is not reported to go above about 0.6, but if in principle it could go up to 1 you want a model fit such that as in A1 predicted or fitted values should remain sensibly bounded.

By eye, one straight line for each species wouldn't seem outrageous, but there is a danger that they stray into negative predictions.

I suggest

S1. Exponential curves of the form $$H = a_j \exp(b_j I)$$ for each species $$j$$ would ensure positive predictions and could be fitted by using a generalized linear model with log link, continuous predictor $$I$$ and species as an indicator variable. Nothing need be transformed. Here is a sketch:

As drawn with parameter guesses to try to match your data, the curves have only slight curvature, which might make the straight-line fits appear competitive.

S2. The left skewness of $$H$$ is a fact but should not determine the form of the model. It's not an assumption of regression or generalized linear models that the marginal distribution of the outcome is, or should be, normal.

S3. The transformation with power 3.16 is a mighty strong transformation (stronger than cubing) and doesn't have a biological rationale. It doesn't work well any way beyond pulling in some lower values. To get a sense of why, consider this plot, which shows that over the interval where most of the data lie, the transformation is close to a straight line function and so can't affect the distribution shape much. As said, its effect is secondary. It's not obvious that we should mess with the data points with low $$H$$ as they are just as important as the others.

S4. You should worry most about a functional fit that makes sense, not most about the residuals.