# Questions tagged [linear-algebra]

A field of mathematics concerned with the study of finite dimensional vector spaces, including matrices and their manipulation, which are important in statistics.

203 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
35 views

33 views

### How to optimize a generalized trace problem in dimensionality reduction

I know how to solve this problem in dimensionality reduction. $argmax_{X}$ $Trace[XLX^T]$ with $XX^T=I$ ,where $L$ is symmetric, $X$ is unitary, and $I$ is identity matrix. But I'd like to know how ...
24 views

### Distribution of the maximum vector (over i.i.d. set) and its dot product with the eigenvectors spanning the rest.

Let $z_1,\dots,z_n$ be i.i.d. draws from $N(0,\Sigma)$, where $\Sigma$ is a $p\times p$ matrix. Assume that $p>n$. Suppose (up to re-labeling) that $z_n=\max_i \|z_i\|_2$. Consider the eigenvectors ...
504 views

### Interpretation of (diagonalized) inverse covariance matrix

There are several threads here about covariance matrix and inverse covariance matrix interpretation (here, here or here). However, I was wondering how to interpret the inverse covariance matrix (or ...
243 views

290 views

### Recommendation system and baseline predictors

I'm participating in programming contest, where I have a data, and where the first number is a user, second number is a movie, and the third is a number in then-points rating. ...
283 views

109 views

### Representing a multivariate normal with a scaled variance

I would like to model an observation to have a multivariate normal distribution but am having some trouble figuring out the linear algebra. So, let us start with a distribution that I know how to ...
91 views

### Laplacian Eigenmaps Derivation Question

I had read a few papers on Laplacian Eigenmaps and have been a bit confused on 1 step in the standard derivation. First I just want to deal with the 1-D case. We are given that we want to find the ...
173 views

### Fitting a linear model with few extreme values

I want to estimate a parameter (let's call it x) by some other paramaters via some linear model. Usually I take lm() in R for such purposes. However, in my situation the parameter x is mostly very ...
112 views

### Can a very bad Coefficient of determination ($R^{2}$) not be indicative of model performance?

Thanks in advance for the advice. I am trying to build a generalized linear model that has many predictors. The $R^{2}$ value of the model is quite low (.21), but when I use the model to predict ...
81 views

### Computing directly comparable wavelet features on variable-length training examples

Consider a classification problem in which the raw data are snippets of a larger 1-D time series signal. In my application, the signal is the response of a motion sensor as a function of time (the raw ...
35 views

### Regression factors and covariance matrix

I am trying to follow someone else's notes. They have two matrices. One is called comfact (company factors). This is a 580 x 5 matrix. The 580 rows represent 580 ...
951 views

### Fast Mahalanobis distance computation with singular covariance matrix

I'm trying to calculate the following Mahlanobis distance. $x^{T}$pinv($C$)$x$ Since covariance matrix, $C$ is singular, pinv($C$) means pseudo-inverse of C. However, my $C$ is very large, so it's ...
6 views

### How to compute a spatial covariance matrix within a cluster?

I'm trying to implement this paper, which is a method to obtain superpixels through a SLIC-based approach, and at some point I need to calculate a spatial covariance matrix for each cluster - or ...
9 views

### Word vectors projection and residual features

I am working on a project that uses word vectors from word2vec. I can come up with semantic feature vectors by subtracting pairs of word vectors, for example I can say a gender vector is formed by ...
18 views

### LDA (linear discriminant analysis) for images: strange eigenvalues

I have the following dataset: I represent each image as a $(67 \times 67, 1)$ vector and add it to the dataframe. df.head() My goal is to determine the ...
45 views

### Finding inverse matrix $(X'X)^{-1}$ with $X$ as design matrix

I'm relatively new to all this and I am trying to figure out how I can derive the matrix $(X'X)^{-1}$ when I have given $x_1, x_2, x_3$ and $y$. $X$ is the design matrix in that case but not sure how ...
31 views

### First Principal Component Direction

I am trying to derive the first principal component direction from the definition and need help in finding which step is going wrong. Here's my attempt: $\mathbf{X} \in \mathbb{R}^{N \times p}$ is the ...
8 views

### How to define data within a 1D matrix as categorical

Is there any way to clearly delineate that the data contained within a matrix is categorical to the reader? Ie: is there a symbol that I can use to mark the data as qualitative and not quantitative.
13 views

### Signs of eigenvectors : Dual PCA

I'm trying to perform a DUAL PCA with numpy, this is are the steps I'm following: 0 - Standardize X, where X is for instance (m,n) 1 - Find eigenvalues, eigenvectors of X.T dot X Plot the projections (...
13 views

### Property of Covariance Matrices and Symmetric Matrices

I have a question about covariance matrices. I have read one interesting property that, all symmetric matrices are diagonalizable. Suppose we have a data matrix $X$ that has only $m$ independent ...
100 views

### @whuber’s solution to generating correlated vector to an existing one

Here https://stats.stackexchange.com/a/313138 @whuber describes a beautiful solution to generating a correlated vector to an existing one. The thing i cant figure out is $SD()$ in following expression:...
26 views

### Covariance and Correlation Matrices

I have a somewhat dumb question. When determining the correlation or covariance (doesn't matter I suppose) amongst random vectors, is the covariance computed among features or among observations? For ...
12 views

### Term for the space describing the relationships between variables?

I'm looking for a term, which I've been referring to as the "data space", though I'm reasonably sure there's a proper term for it: Let's say I've got two variables, and when one doubles, the ...
27 views

### Can someone clearly explain the feature maps, representer theorem and kernels?

I know that we need feature maps for representing non linear function as a linear function. And linear function can be represented as a vector and vectors can be easily manipulated by computer like a ...
I have an idea to help reduce the noise in my signal but am stuck with a significant problem. I have a very noisy data set $y_n[t]; n\in\{0, N_{\text{samples}}-1\}; t\in\{0, T-1\}$ I am fitting this ...