How to z-normalize multi-dimensional time series? z-normalization for 1-dim time series is simple.
$z_i = (x_i-m)/s$
Here, $x_i$ is the element of series for each time index $i$.
$m$ is the mean, and $s$ is the standard deviation.
For n-dim time series, I can estimate $m$ easily, but I don't know how to estimate $s$.
Do I need to estimate covariance matrix and multiply its inverse?
Also, the number of elements of time series is smaller than the dimension of each element.
 A: I guess you need to multiply by the inverse square root:
$$
X\sim N_p(\mu,\Sigma) \Rightarrow Z = \Sigma^{-1/2}(X-\mu) \sim N_p(0_p,I_p)
$$
Now, the concept of a square root of the matrix is not uniquely defined. For a symmetric positive definite matrix, as covariance matrices are, there are two decent choices:


*

*Spectral decomposition (aka principal component analysis) square root:
$$
\Sigma = U \Lambda U', \mbox{ where } \Lambda={\rm diag}(\lambda_1, \ldots, \lambda_p), \lambda_1 \ge \ldots \ge \lambda_p > 0; U'U = UU' = I_p
$$
$$
\Rightarrow \Omega_s = U\Lambda^{-1/2}U', \Lambda^{-1/2} = {\rm diag}(\lambda_1^{-1/2}, \ldots, \lambda_p^{-1/2})
$$
is such square root:
$$
\Omega_s \Sigma \Omega_s = U\Lambda^{-1/2}U' U\Lambda U' U\Lambda^{-1/2}U' = U\Lambda^{-1/2} \Lambda \Lambda^{-1/2} U' = U U' = I_p
$$

*Cholesky decomposition is another popular square root:
$$
\exists \mbox{ lower triangular } L: \Sigma = LL'
$$


My feeling is that Cholesky might be a bit more stable numerically. There are probably other options for square roots, but these appear to be the more popular ones that can be easily found in most computing environments.
A: Usually, you normalize each variable separately, so for each element $i$ you compute its own mean $\mu_i$ and the standard deviation $s_i$ to normalize. This is not a problem with large number of dimensions, as described in this case.
Your suggestion about using the covariance matrix is reasonable too. Take a look at mahalonobis distance, for instance MATLAB's mahal function, which does exactly what you just described, except for the dimension issue.
The dimension issue is a numerical one, see the shrinkage estimation here. You are estimating NxN matrix whereas your data set is NxT and $T<N$. The problem is that your data set is smaller than the number of parameters you want to estimate, i.e. covariances. This will not cause any problems for linear algebra, matrix inverse will work fine. It's just how reliable is the covariance matrix estimate? MATLAB's mahal function will work on your data with no problem reported in most cases.
