# Distribution of Maximum Eigen Value

Suppose I have X, k*n, where $$M=X'X$$. Suppose $$n>>k$$, and $$rank(M) =k-1$$. Suppose $$\lambda_1, \cdots, \lambda_{k-1}$$ are the eigen values of M. Under the assumption that the columns of X are random, what can we say about the distribution of $$\lambda_{max}$$?

Here is my applied problem. Suppose the columns of X are observed values of n random variables. We only have k observations for each variable. I want to know if there exists an underlying variable that can explain them. If there is none, then $$\lambda_i$$ should be close to $$\frac{1}{k-1}$$. I want to know how large $$\lambda_{max}$$ has to be for us to conclude that at least one significant explanatory variable exists. This is similar to the condition number concept, except for the fact that M is not of full rank.

Edit: take n=2000, k=21.

• $M$ isn't full rank?
– Dave
Jan 17 at 0:30
• Yes, it is not.
– deb
Jan 17 at 1:44
• What distributions do the columns of $X$ follow? If all distributions are fair-game, I have written a simulation where two different distributions produce wildly different sets of maximum eigenvalues of $M$.
– Dave
Jan 17 at 2:26
• Assume Gaussian with fat tails
– deb
Jan 17 at 16:17
• What does that mean? Gaussians do not have fat tails (at least not compared to Gaussians). Even different Gaussians are giving me radically different maximum eigenvalue distributions, though.
– Dave
Jan 17 at 16:20

There seems not to be a universal distribution of this maximum eigenvalue. In the simulation below, I draw from $$t_5$$ and $$t_6$$ distributions to create the first twenty columns of $$X$$. Then I create the 21st column by adding some of the first twenty columns. This meets the criteria set out in the question

Even just changing the degrees of freedom in the $$t$$-distribution results in wildly different distributions of the maximum eigenvalue of $$X^TX$$.

library(ggplot2)
set.seed(2024)
N <- 2000
k <- 21
R <- 10000
B <- rbinom(k-1, 1, 0.3)
e1max <- e2max <- rep(NA, R)
for (i in 1:R){
X1 <- matrix(rt(N*(k-1), 5), N, k-1)
Xk1 <- X1 %*% B
X1 <- cbind(X1, Xk1)
M1 <- t(X1) %*% X1
e1 <- eigen(M1)
e1max[i] <- max(e1$values) X2 <- matrix(rt(N*(k-1), 6), N, k-1) Xk2 <- X2 %*% B X2 <- cbind(X2, Xk2) M2 <- t(X2) %*% X2 e2 <- eigen(M2) e2max[i] <- max(e2$values)
}
d1 <- data.frame(
Eigenvalue = e1max,
Distribution = "t5"
)
d2 <- data.frame(
Eigenvalue = e2max,
Distribution = "t6"
)
d <- rbind(d1, d2)
ggplot(d, aes(x = Eigenvalue, fill = Distribution)) +
geom_density(alpha = 0.25)


(This took a few minutes to run, but the time can be decreased by decreasing the number of repetitions R.)

If you want to say something about the distribution of maximum eigenvalues in your particular situation, your best bet might be to bootstrap the rows of $$X$$, multiply to give some kind of bootstrapped matrix $$M_{\text{boot}}$$, and calculate the maximum eigenvalue of that boostrapped matrix. I give an example below.

library(ggplot2)
set.seed(2024)
N <- 2000
k <- 21
R <- 10000
B <- rbinom(k-1, 1, 0.3)

X <- matrix(rt(N*(k-1), 5), N, k-1)
Xk <- X %*% B
X <- cbind(X, Xk)
emax <- rep(NA, R)
for (i in 1:R){

idx <- sample(seq(1, N, 1), N, replace = T)
X_boot <- X[idx, ]
M_boot <- t(X_boot) %*% X_boot
e <- eigen(M_boot)
emax[i] <- max(e\$values)

}
d <- data.frame(Eigenvalue = emax, Distribution = "Bootstrap")
ggplot(d, aes(x = Eigenvalue, fill = Distribution)) +
geom_density(alpha = 0.25)