# Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

620 questions
Filter by
Sorted by
Tagged with
16 views

### How to calculate ${w}^\top \mathrm{\Sigma} w$? [closed]

I have: A vector $w_{2\times1} = [[w_1], [w_2]]$ A matrix $\mathrm{R_{2\times21}} = \left[ {\begin{array}{ccccc} [\mathrm{R_1}], [\mathrm{R_2}] \end{array} } \right]$ where $\mathrm{R_1}$ and ...
16 views

### Does invariance of PCA under orthogonal transformation hold for data that is not centered?

I read the proof in the top answer to this question, but that page assumes that $\overline{A} = 0$. If the data instead has some nonzero mean $\mu$, I'm not sure if the same logic applies: ...
1 vote
27 views

### I am not able to understand how did the elementwise multiplication came into the picture of backpropagation in neural networks

I have understood the backpropagation algorithm along with the chain rule well enough that I can derive it on my own, but I don't understand where the elementwise multiplication came from and how does ...
7 views

### Transformation to linearize dataset [duplicate]

How can I transform the following dataset to a more linear representation? R code: ...
• 1,122
1 vote
20 views

### Is it possible to apply the kernel trick to a "mahalnobis distance learner" such as GLS?

1.https://arxiv.org/pdf/0804.1441.pdf 2.https://www.sciencedirect.com/science/article/abs/pii/S0925231210001165 These papers describe kernelizing a mahalanobis distance learner. I am interested in ...
10 views

### Sum of multiple covariance matrices looks like identity matrix

Suppose $X$ and $Y$ are two $a \times b$ matrices, randomly sampled from the same normal distribution. I found an interesting phenomenon: If we sum $X X^T$ multiple times, each time $X$ is randomly ... 40 views

1 vote
55 views

### Variance calculation in matrix notation for $var(z-Ax)$

I noted from a post here that $$var(z - \mathbf{A}x)=var(z)+var(\mathbf{A}x)-\mathbf{A}cov(z, -x)-cov(z,-x)\mathbf{A}^T (Eq. 1)$$ (I dropped the conditional part in the original formula from the post ...
• 197
1 vote
36 views

### Why does PCA maximize variance between the standard deviations?

Consider an $n \times n$ covariance matrix $\Sigma$ (so semi positive-definite, symmetric and realvalued). We can find the $n$ principle components by $n$ times finding the direction of maximum ...
1 vote
13 views

24 views

### Derivative of quadratic form of vector-valued function

This seems like a trivial question but I am currently stuck and cannot see what I am doing wrong. So let us consider a function $f(x) : \mathbb{R}^d \rightarrow \mathbb{R}^d$. I want to compute the ...
44 views

### Relation between generalization bounds of Kernel Ridge Regression and largest eigenvalue of the kernel Gram matrix

Consider a positive-definite, symmetric function $k(x_1, x_2)$ which is used, given the dataset $\{(x_i, y_i)\}_{i=1}^m$, to construct the Gram matrix $K = [k(x_i, x_j)]_{i,j \in 1, ..., m}$. What is ...
• 31
14 views

### How one actually explores data with linear algebra when matrices are just transformations?

I am a beginner in ML and I don't quite understand how we can use linear algebra to extract information from our data matrix. For me matrices only have meaning as a transformation between vector ...
• 61
### Hows does coefficient $b_1$ change when estimating $b_1 x_1+b_2 x_2+b_3 x_3$ instead of $b_1 x_1+b_2 x_2$
My question is related to ,  and . Assume we estimate a multiple regression, $$y = a + b_1x_1 + b_2x_2 + u$$ and are mainly interested in the value of $\hat{b}_1$ (lets denote this specific ...