18
$\begingroup$

I studied mathematics a decade ago, so I have a math and stats background, but this question is killing me.

This question is still a bit philosophical to me. Why did statisticians develop all sort of techniques in order to work with random matrices? I mean, didn't a random vector solve the problem? If not, what is the mean of the diferent columns of a random matrix? Anderson (2003, Wiley) considers a random vector a special case of a random matrix with only one column.

I don't see the point of having random matrices (and I'm sure that's because I'm ignorant). But, bear with me. Imagine I have a model with 20 random variables. If I want to compute the joint probability function, why should I picture them as a matrix instead of a vector?

What am I missing?

ps: I'm sorry for the poorly tagged question, but there were no tags for random-matrix and I can't create one yet!

edit: changed matrix to matrices in the title

$\endgroup$
  • 1
    $\begingroup$ I think you're fine conceptually thinking about it as a random vector that has been rearranged so that it's matrix. $\endgroup$ – Matthew Gunn Nov 3 '16 at 18:30
  • 2
    $\begingroup$ Possibly relevant: en.wikipedia.org/wiki/Random_projection. $\endgroup$ – dsaxton Nov 3 '16 at 18:38
  • 9
    $\begingroup$ You might as well ask why matrices are of any interest. It is perfectly natural to view as random any matrix used to represent a phenomenon observed or measured in the real world. This results in a plethora of possible types and models for random matrices, ranging from adjacency matrices of random graphs to sample covariance matrices and more. $\endgroup$ – whuber Nov 3 '16 at 19:07
  • 2
    $\begingroup$ @Aksakal I think the OP's point is when is it useful to analyse something as random matrices. eg in image classification you typically turn your image matrices into vectors..there is no matrix 'analysis'. so whuber's comment is the best answer so far: eg a covariance matrix has to be positive semi definite - if you want to simulate random covariance matrices its easier to work with a matrix specification than a vector. $\endgroup$ – seanv507 Nov 3 '16 at 20:57
  • 2
    $\begingroup$ Random matrixes are just a special case of random tensors. $\endgroup$ – Anony-Mousse Nov 3 '16 at 22:55
23
$\begingroup$

It depends which field you're in but, one of the big initial pushes for the study of random matrices came out of atomic physics, and was pioneered by Wigner. You can find a brief overview here. Specifically, it was the eigenvalues (which are energy levels in atomic physics) of random matrices that generated tons of interest because the correlations between eigenvalues gave insight into the emission spectrum of nuclear decay processes.

More recently, there has been a large resurgence in this field, with the advent of the Tracy-Widom distribution/s for the largest eigenvalues of random matrices, along with stunning connections to seemingly unrelated fields, such as tiling theory, statistical physics, integrable systems, KPZ phenomena, random combinatorics and even the Riemann Hypothesis. You can find some more examples here.

For more down-to-earth examples, a natural question to ask about a matrix of row vectors is what its PCA components might look like. You can get heuristic estimates for this by assuming the data comes from some distribution, and then looking at covariance matrix eigenvalues, which will be predicted from random matrix universality: regardless (within reason) of the distribution of your vectors, the limiting distribution of the eigenvalues will always approach a set of known classes. You can think of this as a kind of CLT for random matrices. See this paper for examples.

$\endgroup$
  • $\begingroup$ Thanks for your answer. I suspected it could be linked to some mathematical technicalities. Bur now it all makes sense, the physics has always been around math and the other way around too. $\endgroup$ – Eduardo Nov 4 '16 at 13:18
  • $\begingroup$ Wigner's paper was in mid 1950s, Wishart's paper was in late 1920's. So, it's hard to claim that Wigner gave the first big push, in my opinion. $\endgroup$ – Aksakal Nov 4 '16 at 15:18
13
$\begingroup$

You seem to be comfortable with applications of random vectors. For instance, I deal with this kind of random vectors every day: interest rates of different tenors. Federal Reserve Bank has H15 series, look at Treasury bills 4-week, 3-month, 6-month and 1-year. You can think of these 4 rates as a vector with 4 elements. It's quire random too, look at the historical values on the plot below.

enter image description here

As with any random numbers we might ask ourselves: what's the covariance between them? Now you get 4x4 covariance matrix. If you estimate it on one month daily data, you get 12 different covariance matrices each year, if you want them non-overlapping. The sample covariance matrix of random series is itself a random object, see Wishart's paper "THE GENERALISED PRODUCT MOMENT DISTRIBUTION IN SAMPLES FROM A NORMAL MULTIVARIATE POPULATION." here. There's a distribution called after him.

This is one way to get to random matrices. It's no wonder that the random matrix theory (RMT) is used in finance, as you can see now.

$\endgroup$
  • 1
    $\begingroup$ You mention estimating it on "one month data", do you mean "one month's worth of daily data"? $\endgroup$ – John Nov 3 '16 at 18:58
  • 1
    $\begingroup$ @John, in this particular example, yes, daily. However, some series are measured intra day, such as equity prices. If you have a portfolio of stocks, you can get intra day covariance matrices at very high frequencies. $\endgroup$ – Aksakal Nov 3 '16 at 19:00
  • 3
    $\begingroup$ +1. Paraphrasing here, but I think one way to motivate random matrices is that in these cases, we are often interested in the relation between (not just within) random vectors. This naturally motivates thinking about matrices instead of vectors. $\endgroup$ – Cliff AB Nov 3 '16 at 19:01
  • $\begingroup$ @Aksakal The data you mention comes in a monthly frequency as well as daily. I found it confusing before you revised the post because it wouldn't make sense to estimate a covariance matrix using one month of monthly data. $\endgroup$ – John Nov 3 '16 at 19:05
  • $\begingroup$ @John, you could estimate the covariance matrix on monthly or weekly data, overlapping or non-overlapping etc. There's no one right way of doing this. $\endgroup$ – Aksakal Nov 3 '16 at 19:48
8
$\begingroup$

In theoretical physics random matrices play an important role to understand universal features of energy spectra of systems with particular symmetries.

My background in theoretical physics may cause me to present a slightly biased point of view here, but I would even go so far to suggest that the popularity of random matrix theory (RMT) originated from its successful application in physics.

Without going too much into detail, for example energy spectra in quantum mechanics can be obtained by calculating eigenvalues of the systems Hamiltonian - which can be expressed as an hermitian matrix. Often physicists are not interested in particular systems but want to know what are the general properties of quantum systems that have chaotic properties, which leads the values of the hermitian Hamiltonian matrix to fill the matrix-space ergodically upon variation of the energy or other parameters (e.g. boundary conditions). This motivates treating a class of physical systems as random matrices and looking at average properties of these systems. I recommend literature on the Bohigas-Gianonni-Schmidt conjecture if you want to dive into this deeper.

In short, one can for instance show that energy levels of systems that have time reversal symmetry behave universally different than energy levels of systems which have no time reversal symmetry (which happens for instance if you add a magnetic field). An in fact quite short calculation using Gaussian random matrices can show that the energy levels tend to be differently close in both systems.

These results can be extended and helped to understand also other symmetries, which had a major impact on different fields, like also particle physics or the theory of mesoscopic transport and later even in financial markets.

$\endgroup$
  • 1
    $\begingroup$ this is why I quit physics :) too much "brain damage" $\endgroup$ – Aksakal Nov 3 '16 at 20:33
  • $\begingroup$ Statisticians like Wishart where before the atomic physicists. $\endgroup$ – kjetil b halvorsen Nov 4 '16 at 9:09
5
$\begingroup$

A linear map is a map between vector spaces. Suppose you have a linear map and have chosen bases for its domain and range spaces. Then you can write a matrix which encodes the linear map. If you want to consider random linear maps between those two spaces, you should come up with a theory of random matrices. Random projection is a simple example of such a thing.

Also, there are matrix/tensor valued objects in physics. The viscous stress tensor is one such (among a veritable zoo). In a nearly homogeneous viscoelastic materials, it can be useful to model the strains (elastic, viscous, et al.) and hence the stresses pointwise as a random tensor with small variance. Although there is a "linear map" sense to this stress/strain, it is more honest to describe this application of random matrices as randomizing something that was already a matrix.

$\endgroup$
4
$\begingroup$

Compressive sensing as an application in image processing relies on random matrices as combined measurements of a 2D signal. Specific properties of these matrices, namely coherence, are defined for these matrices and play a role in the theory.

Grossly simplified, it turns out that minimizing the L1 norm of a certain product of a Gaussian matrix and an sparse input signal allows you to recover much more information than you might expect.

The most notable early research in this area I know of is Rice University's work: http://dsp.rice.edu/research/compressive-sensing/random-matrices

The theory of matrix products as "measurements of a signal" goes at least as far back as WW2. As a former professor of mine recounted to me, individually testing every army enlistee for, say, syphilis, was cost prohibitive. Mixing together these samples in a systematic way (by mixing portions of each blood sample together and testing them) would reduce the number of times a test needed to be performed. This could be modeled as a random binary vector multiplied with a sparse matrix.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.