13
$\begingroup$

How did the hat matrix get its name

$\hat{\mathbf{H}} = \mathbf{X} \left( \mathbf{X}^\textsf{T} \mathbf{X} \right)^{-1} \mathbf{X}^\textsf{T}$

I am interested in the etymology of the term. Who gave it a name and why?

$\endgroup$
4

2 Answers 2

18
$\begingroup$

The name "hat matrix" is a mnemonic: a shortcut to help us remember the role it plays in regression. As @RobertLong explains in Learning hat matrix,

The hat matrix is the projection matrix that maps the response vector $Y$ to the vector of fitted values $\hat{Y}$ (hence the name "hat" matrix).

As to history, according to

David, H. A. “First (?) Occurrence of Common Terms in Mathematical Statistics.” The American Statistician, vol. 49, no. 2, 1995, pp. 121–33. https://doi.org/10.2307/2684625.

the term "hat matrix" first appears in

Hoaglin, David C., and Roy E. Welsch. “The Hat Matrix in Regression and ANOVA.” The American Statistician, vol. 32, no. 1, 1978, pp. 17–22. https://doi.org/10.2307/2683469.

But the authors themselves attribute it to J. W. Tukey.

$\endgroup$
8
  • $\begingroup$ It's funny that the name was meant as a mnemonic to remember the purpose, yet the purpose is remembered while the name is unclear. When the shortcut is longer than the regular route... $\endgroup$
    – Jason C
    Commented Nov 2, 2022 at 10:29
  • 2
    $\begingroup$ @JasonC The idea to describe it as a mnemonic was mine. The Hoaglin and Welsch paper only says: "The term hat matrix is due to John W. Tukey who introduced us to the technique about ten years ago." So we don't know Tukey's purpose; maybe he used it in jest also? But the definition of mnemonic is pretty broad, so I think it qualifies. $\endgroup$
    – dipetkov
    Commented Nov 2, 2022 at 11:14
  • $\begingroup$ It's an appropriate name in any case! $\endgroup$
    – Jason C
    Commented Nov 2, 2022 at 14:03
  • 1
    $\begingroup$ @JasonC Since this question is in part about history, I'll assume that this might be of interest to the OP: John Tukey appears in a couple of amusing episodes in Richard Feynman's biography Genius. They were in Princeton together. One story is about hexahexaflexagons... $\endgroup$
    – dipetkov
    Commented Nov 2, 2022 at 14:27
  • $\begingroup$ I remembered this amusing question while trying out ChatGPT (after reading yet one more article about how cool the technology is). So here its response: $\endgroup$
    – dipetkov
    Commented Apr 19, 2023 at 21:38
17
$\begingroup$

In fact the formula should read $\mathbf{H} = \mathbf{X} \left( \mathbf{X}^\textsf{T} \mathbf{X} \right)^{-1} \mathbf{X}^\textsf{T}$, therefore $\hat{\mathbf{y}}=\mathbf{H}\mathbf{y}$. The hat matrix is called hat matrix because it puts a hat on the $\mathbf{y}$. (This is in fact mentioned on the Wikipedia page linked by @dipetkov, but I had heard it before from somebody who I think had heard Tukey mentioning it.)

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.