7
$\begingroup$

I have McElreath's book (Statistical Rethinking) and was intrigued by his callout against "Histomancy" (see image below).

Histomancy

I felt the section a bit wanting and I am left unsure as to what is proposed we do then. I even went a little further and watched his lecture videos, hoping for additional information on the matter.

My only takeaway is that we should instead use our knowledge about the outcome variable and ignore the histograms. For example, say we have a count variable and so then we know we must be dealing with a Poisson or a related likelihood. Then it comes down to comparing models that could feasibly express the likelihood and see which one fits the data best.

Is there anything else that we can use besides the basic knowledge that (ah, I have a count variable, Poisson!) to better understand where we should start with our guess for what distribution likely generated our data?

$\endgroup$

1 Answer 1

13
$\begingroup$

McElreath appears to be maligning the oft repeated practice of requiring regressors (to the right of the equality sign, sometimes 'independent' or 'predictor' variables) and regressands (to the left of the equality sign, sometimes 'dependent' or 'outcome' variables) to be normally distributed (i.e. "Gaussian") in the context of something like OLS regression.

In fact none of these variable needs to be normally, or even nearly normally distributed. The residuals require this assumption, as in the simple model here:

$$y_i = \beta_0 + \beta_x x_i + \varepsilon_i\text{; where }\varepsilon \sim \mathcal{N}(0,\sigma)$$

It is relatively easy to demonstrate this:

    n <- 200
    x <- runif(n)
    b0 <- 10
    bx <- -2
    s <- 0.1
    e <- rnorm(n,0,s)
    y <- b0 + bx*x + e
    summary(lm(y~x))
    hist(y)
    hist(x)
    hist(e)

Notice that:

  1. You quite adequately estimate $\beta_0$, $\beta_x$, and $\sigma$ using the OLS MLE estimators:
          Coefficients:
                      Estimate  Std. Error t value  Pr(>|t|)    
          (Intercept) 10.00007    0.01337  747.85   <2e-16 ***
          x           -2.00687    0.02235  -89.81   <2e-16 ***

          Residual standard error: 0.0957 on 198 degrees of freedom
  1. The histograms of $y$ and $x$ are nothing like normally distributed:

Histogram of y Histogram of x

  1. The histogram of $\varepsilon$ is approximately normal:

Histogram of residuals

Of course there are other linear regression models than OLS (including multiple regression), but MLE estimation is quite often used for such models, and the conflation of distributions of variables with residuals is reflected widely in questions on this site, in the literature, and in research meetings.

The upshot is that we should strive to understand our modeling assumptions (whether our data are continuous, count, or what have you) in application rather than waste time in pointless efforts (i.e. "Histomancy") like normalizing all our regression variables.

$\endgroup$
2
  • $\begingroup$ Thanks for the answer and the OLS makes sense. But what about beyond OLS? What do we do we do then? Just know that we have a binary variable so then we must be dealing with the Bernoulli distribution. $\endgroup$
    – John Stud
    Mar 24, 2021 at 16:23
  • $\begingroup$ @JohnStud Ask a separate question, that is what the website is for. :) I answered the question about "Histomancy." BTW: Welcome, to CV John Stud. $\endgroup$
    – Alexis
    Mar 25, 2021 at 16:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.