# Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

680 questions
Filter by
Sorted by
Tagged with
1 vote
36 views

### how to approximate the eigendecomposition of a correlation matrix when the data have been standardized?

Context I am working to develop a penalized regression framework that will scale up to analyzing high dimensional data with a certain correlation structure. Let $X$ represent an $n \times p$ matrix of ...
13 views

### Prove that if eigenvalues are all different then eigenvectors are linearly independent [migrated]

I was trying to work out a proof of the fact that if eigenvalues $\lambda_1$, $\lambda_2$,...,$\lambda_n$ are all different, then the eigenvectors $\mathbf{v}_1$, $\mathbf{v}_2$,...,$\mathbf{v}_n$ are ...
47 views

### Why do OLS libraries fit models using the MP Pseudoinverse of the design matrix?

For the linear model $y = X\beta$ for design matrix $X$, it's well known that the optimal solution is $\hat{\beta} = (X'X)^{-1}X'y$. Some statistical libraries (such as Python's statsmodels) estimate ...
• 612
20 views

### Constrained Cholesky Decomposition

Suppose that I have an $(n\times 1)$ vector of random variables, $\varepsilon$. However, I know that $k$ linear combinations of $\varepsilon$ are 0. Specifically, I know that for a $(k\times n)$ ...
182 views

### Rasmussen Equation 5.9

Can any one add the steps showing how Rasmussen (Gaussian Processes for Machine Learning, the MIT Press, 2006) got from line 1 to line 2 of equation 5.9. (pg 114)? It is calculating the gradient of ...
• 331
282 views

### Does the conditional expectation operator have an interpretable decomposition like the projection matrix does in linear algebra?

I'm trying to draw a parallel between the concept of projections in a finite linear space to an infinite linear space. Here is the set-up, first in the finite dimensional case, and then second in the ...
70 views

### The Math Behind the Conditional Probability of a Probabilistic PCA

I am trying to understand how to calculate the conditional distribution of probabilistic principal component analysis. This is explained in the book "Pattern Recognition and Machine Learning"...
• 75
1 vote
13 views

### Positive distance weighting

I have an overdetermined linear system of equations that's solved with least squares. I'd like to weight the equations to penalize a bunch of inputs clumped up together. Ideally if two (or more) ...
1 vote
25 views

### What exactely is "the part of the interaction orthogonal to factors $A$ and $B$" in a two-way ANOVA?

Consider a two-way ANOVA with factors $A$ and $B$ and the interaction $A\times B$. The author of this answer answer https://stats.stackexchange.com/a/608301/359647 (@svendvn) explains that the Type ...
• 128
17 views

### Error term in SGD with momentum

I am reading the article "How Momentum really works" (https://distill.pub/2017/momentum/), and i am confused in one point: I am trying to derive the convergence rate for momentum from the ...
• 121
10 views

• 345