# Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

203 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1answer
35 views

0answers
33 views

### How to optimize a generalized trace problem in dimensionality reduction

I know how to solve this problem in dimensionality reduction. $argmax_{X}$ $Trace[XLX^T]$ with $XX^T=I$ ,where $L$ is symmetric, $X$ is unitary, and $I$ is identity matrix. But I'd like to know how ...
0answers
24 views

### Distribution of the maximum vector (over i.i.d. set) and its dot product with the eigenvectors spanning the rest.

Let $z_1,\dots,z_n$ be i.i.d. draws from $N(0,\Sigma)$, where $\Sigma$ is a $p\times p$ matrix. Assume that $p>n$. Suppose (up to re-labeling) that $z_n=\max_i \|z_i\|_2$. Consider the eigenvectors ...
0answers
504 views

### Interpretation of (diagonalized) inverse covariance matrix

There are several threads here about covariance matrix and inverse covariance matrix interpretation (here, here or here). However, I was wondering how to interpret the inverse covariance matrix (or ...
0answers
243 views

0answers
290 views

### Recommendation system and baseline predictors

I'm participating in programming contest, where I have a data, and where the first number is a user, second number is a movie, and the third is a number in then-points rating. ...
0answers
283 views

0answers
109 views

### Representing a multivariate normal with a scaled variance

I would like to model an observation to have a multivariate normal distribution but am having some trouble figuring out the linear algebra. So, let us start with a distribution that I know how to ...
0answers
91 views

### Laplacian Eigenmaps Derivation Question

I had read a few papers on Laplacian Eigenmaps and have been a bit confused on 1 step in the standard derivation. First I just want to deal with the 1-D case. We are given that we want to find the ...
0answers
173 views

### Fitting a linear model with few extreme values

I want to estimate a parameter (let's call it x) by some other paramaters via some linear model. Usually I take lm() in R for such purposes. However, in my situation the parameter x is mostly very ...
0answers
112 views

### Can a very bad Coefficient of determination ($R^{2}$) not be indicative of model performance?

Thanks in advance for the advice. I am trying to build a generalized linear model that has many predictors. The $R^{2}$ value of the model is quite low (.21), but when I use the model to predict ...
0answers
81 views

### Computing directly comparable wavelet features on variable-length training examples

Consider a classification problem in which the raw data are snippets of a larger 1-D time series signal. In my application, the signal is the response of a motion sensor as a function of time (the raw ...
0answers
35 views

### Regression factors and covariance matrix

I am trying to follow someone else's notes. They have two matrices. One is called comfact (company factors). This is a 580 x 5 matrix. The 580 rows represent 580 ...
1answer
951 views

### Fast Mahalanobis distance computation with singular covariance matrix

I'm trying to calculate the following Mahlanobis distance. $x^{T}$pinv($C$)$x$ Since covariance matrix, $C$ is singular, pinv($C$) means pseudo-inverse of C. However, my $C$ is very large, so it's ...
0answers
6 views

### How to compute a spatial covariance matrix within a cluster?

I'm trying to implement this paper, which is a method to obtain superpixels through a SLIC-based approach, and at some point I need to calculate a spatial covariance matrix for each cluster - or ...
0answers
9 views

### Word vectors projection and residual features

I am working on a project that uses word vectors from word2vec. I can come up with semantic feature vectors by subtracting pairs of word vectors, for example I can say a gender vector is formed by ...
0answers
18 views

### LDA (linear discriminant analysis) for images: strange eigenvalues

I have the following dataset: I represent each image as a $(67 \times 67, 1)$ vector and add it to the dataframe. df.head() My goal is to determine the ...
0answers
45 views

### Finding inverse matrix $(X'X)^{-1}$ with $X$ as design matrix

I'm relatively new to all this and I am trying to figure out how I can derive the matrix $(X'X)^{-1}$ when I have given $x_1, x_2, x_3$ and $y$. $X$ is the design matrix in that case but not sure how ...
0answers
31 views

### First Principal Component Direction

I am trying to derive the first principal component direction from the definition and need help in finding which step is going wrong. Here's my attempt: $\mathbf{X} \in \mathbb{R}^{N \times p}$ is the ...
0answers
8 views

### How to define data within a 1D matrix as categorical

Is there any way to clearly delineate that the data contained within a matrix is categorical to the reader? Ie: is there a symbol that I can use to mark the data as qualitative and not quantitative.
0answers
13 views

### Signs of eigenvectors : Dual PCA

I'm trying to perform a DUAL PCA with numpy, this is are the steps I'm following: 0 - Standardize X, where X is for instance (m,n) 1 - Find eigenvalues, eigenvectors of X.T dot X Plot the projections (...
0answers
13 views

### Property of Covariance Matrices and Symmetric Matrices

I have a question about covariance matrices. I have read one interesting property that, all symmetric matrices are diagonalizable. Suppose we have a data matrix $X$ that has only $m$ independent ...
0answers
100 views

### @whuber’s solution to generating correlated vector to an existing one

Here https://stats.stackexchange.com/a/313138 @whuber describes a beautiful solution to generating a correlated vector to an existing one. The thing i cant figure out is $SD()$ in following expression:...
1answer
26 views

### Covariance and Correlation Matrices

I have a somewhat dumb question. When determining the correlation or covariance (doesn't matter I suppose) amongst random vectors, is the covariance computed among features or among observations? For ...
0answers
12 views

### Term for the space describing the relationships between variables?

I'm looking for a term, which I've been referring to as the "data space", though I'm reasonably sure there's a proper term for it: Let's say I've got two variables, and when one doubles, the ...
1answer
27 views

### Can someone clearly explain the feature maps, representer theorem and kernels?

I know that we need feature maps for representing non linear function as a linear function. And linear function can be represented as a vector and vectors can be easily manipulated by computer like a ...
0answers
27 views

### Reverse-Mode Automatic Differentiation with respect to a Matrix: How to “Matrix Multiply” 4D Tensors?

This is a follow up question I have on this excellent answer: https://stats.stackexchange.com/a/235758/307400. I will save me writing down any details about reverse-mode automatic differentiation, the ...
0answers
49 views

### Using eigenvalues of the covariance matrix to reduce noise in my data

I have an idea to help reduce the noise in my signal but am stuck with a significant problem. I have a very noisy data set $y_n[t]; n\in\{0, N_{\text{samples}}-1\}; t\in\{0, T-1\}$ I am fitting this ...