# Usage of tensor notation in statistics

A friend of mine (mathematician) basically told me I shouldn't bother with matrix algebra and should focus on tensor analysis/manipulation. He said it's much more general and intuitive. I've been trying to grasp most of the matrix literature (Harville, Magnus, Gentle) due to its relevance to linear models.

My question is this: is there a part of mathematical statistics/prob. theory where tensor notation is employed regularly? If so, why isn't it popular elsewhere since it (supposedly) makes matrix manipulations easier?

• Have you read stats.stackexchange.com/questions/198061/… ? – Tim Dec 21 '16 at 12:28
• Intuitive supposedly means easy to learn. In my experience when used as a sales pitch it usually means easy to remember and use once you understand it, which is quite different, as in "the syntax of this software is highly intuitive". It usually isn't! I distrust all such claims generically. Your friend is very comfortable with tensors; that's great and you could be too if you worked hard enough with them. – Nick Cox Dec 21 '16 at 13:18
• @Tim I believe that thread might represent exactly the opposite of what the mathematician friend is advocating. The contrast is between manipulation of objects with multiple indices versus manipulation of more abstract mathematical objects using more abstract algebraic rules. Another way to understand the distinction is that people who need to think about linear algebraic objects tend to use basis-free methods (without indices) whereas those who need to compute with them (statisticians, computer programmers, physicists, engineers) eventually are forced to use indices. – whuber Dec 21 '16 at 15:34
• Very good comment above, @whuber (+1). – amoeba Feb 15 '17 at 9:49

The most obvious and straightforward application of tensors (that I know of) in statistics is computing high-order moments of a multivariate distribution. For example, consider a random vector $x\sim F$, where $F$ is some $p$-dimensional distribution. Given some data matrix $X \in \mathbb{R}^{n\times p}$ where $n$ is the number of observations, each of which is drawn iid from $F$, the second moment $\mathbb{E}(xx^\top) = \mathbb{E}(x\otimes x)$ can be estimated from the sample $X$ as follows $$\hat{\mathbb{E}}(x\otimes x) = \frac1n \sum_{i=1}^n X_{i\cdot} \otimes X_{i\cdot} = \frac1n X^\top X \in \mathbb{R}^{p\times p}$$ where $X_{i\cdot}$ is the $i^{th}$ row of $X$. Certainly this is a matrix that is only a few operations away from the covariance matrix. Continuing on to the third moment, which is again related to "co-skewness," we see we are dealing with an order-3 tensor $$\hat{\mathbb{E}}(x\otimes x \otimes x) = \frac1n \sum_{i=1}^n X_{i\cdot} \otimes X_{i\cdot}\otimes X_{i\cdot} \in \mathbb{R}^{p\times p\times p}$$ The "co-kurtosis" tensor is order 4 and so on for higher-order moments.