Questions tagged [matrix-decomposition]

Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.

Filter by
Sorted by
Tagged with
1 vote
0 answers
16 views

Eckart–Young–Mirsky theorem for $n \gg m$

It has been proven that the best reconstruction error in the $k$ rank matrix estimation problem in terms of Frobenius or $L2$ norm is given by the $k$-truncated SVD as shown here. I've read in ...
1 vote
0 answers
53 views

Why PCA is invariant under rotation

Lets say that we have a matrix of variables (the columns are variables and rows are the observations) called X whenre X = [x1, x2, ...., xp] where ...
0 votes
0 answers
17 views

Generate dataset that adhere to a given correlation matrix [duplicate]

I'm newbie in statistics and this question might be naive so kindly excuse me in advance. I have a correlation matrix of 40 features and I want to generate a dataset of hundreds of observations that ...
  • 101
1 vote
0 answers
30 views

Decomposition analysis for data between zero and one

I want to analyze latent components of data that has values between zero and one (including zero and one). In detail, the data structure is n x m and I'm looking to find the r underlying components. ...
  • 31
0 votes
0 answers
41 views

How to find a decomposition of multivariate X along which y varies the most?

I'm looking for an existing algorithm which carries out the task shown in the title. My use-case in other words: I have a set of continuous independent variables (X) and a continuous dependent ...
  • 113
1 vote
0 answers
18 views

quasi-PCA reconstruction of the matrix by orthogonal basis

Let's say I have a "data" matrix $X$ of $N$ rows and $p$ cols with $N \gg p$. Now PCA with $L$ components can be formulated as $$ X_L = \underset{Y:rank(Y) = L}{\text{argmin}} ||X- Y||^2_F, ...
0 votes
0 answers
26 views

The relationship between eigenvalues of a covariance matrix and the variances of the same data matrix after using eigenvectors as bases

Suppose we have a data matrix $\mathbf{X}\in \mathbb{R}^{M\times N}$ with $M$ features, $N$ samples and zero means ($M \lt N$). The covariance matrix of $\mathbf{X}$ is $\mathbf{C_x}=\frac{1}{N}\...
  • 1
0 votes
0 answers
38 views

Decomposing Distance Matrix D for approximating Original Matrix A

Let's say we have a matrix $A \in R^{n \times d}$ where n is the number of elements and d is the dimension size. And we calculate the pairwise distances between each elements; say cosine for instance ...
0 votes
0 answers
37 views

Find the principal component and the proportion of the total population variance explained by each when the variance covariance matrix is given

I can understand the part where we have to find the principal component from the variane covariance matrix- find eigen values, make eigen vector and normalise. The principal component would be ...
1 vote
1 answer
71 views

Matrix dimensions in Linear Algebra vs Time series Analysis

I am confused or may misunderstand the dimensions of a Matrix when I was reading about time series analysis. From what I understand in linear Algebra, if we have a Matrix $A \in \mathbf{R}^{m*n}$, ...
  • 13
0 votes
0 answers
221 views

Funk SVD for binary data - product like or dislike

Assume the following situation: you have a user-item sparse matrix. However, instead of the usual 1 to 5 rating scale, items can only receive a positive (1) or negative (-1) feedback. Thus, the matrix ...
  • 143
0 votes
0 answers
9 views

Can a covariance matrix be decomposed into a single vector of meaningful distances?

I am interested in creating a single vector to describe a covariance matrix such that the most dissimilar variables are the most extreme values and the other variables have some numeric distance from ...
1 vote
0 answers
8 views

CP Tensor Decomposition and Correlating Sample Magnitudes with Variables of Interest

I am learning about tensor decomposition, specifically CP, and am trying to understand if I can use it for my research. To give a bit more detail, I have brain imaging data from 10 participants, with ...
  • 315
0 votes
0 answers
14 views

Proof of SVD generates equal eigenvectors as PCA [duplicate]

Figure link: https://people.cs.pitt.edu/~milos/courses/cs3750-Fall2014/lectures/class9.pdf In process of PCA, we either decompose covariance matrix, or do SVD on X. $$ C = \frac{1}{n-1} X^T X = \frac{...
  • 1
0 votes
0 answers
36 views

What is the correlation between random variables after being multiplied by the same lower triangle matrix decomposed from a covariance matrix?

Assume $C_{n \times n}$ is a positive, symmetric and semi-definite covariance matrix, we know that the LU decomposition exists, i.e., $C_{n \times n}=L_{n \times n}U_{n \times n}$. Now $n$ ...
0 votes
0 answers
46 views

Constrained Matrix Decomposition

I am working on a structural vector autoregression that requires imposing constraints on a matrix factorization. In particular, I have an N-dimensional positive definite matrix $\Sigma$ that I need to ...
  • 2,147
0 votes
0 answers
32 views

Why does the ridge penalty shrink the singular values? [duplicate]

I am trying to understand the following analysis of ridge regression. I am new to SVD but I think I have a sufficient grasp on most of the content. There are two things I am struggling with. The ...
  • 531
1 vote
1 answer
37 views

How to decompose a random walk (array) into its Markov Chain transition matrix?

The algorithm, PageRank, receives a Markov Chain transition matrix (page links from one to another.) Either by random walk, or more efficiently, eigenvectors, the stationary distribution of the Markov ...
  • 1,728
1 vote
1 answer
35 views

Is the first independent component of independent component analysis always important?

I was looking at a neuroscience paper that used ICA to reduce dimensionality of calcium signaling profiles in 20 randomly selected neurons of a zebrafish brain. I presume that in Figure 2, ICA was ...
  • 11
0 votes
0 answers
19 views

Why does Non-Negative Matrix Factorization reconstructs exactly the same matrix?

I'm trying recently to get into recommender systems and almost all tutorials I find mention collaborative filtering done with matrix factorization. I found this tutorial that describes how to build ...
1 vote
0 answers
23 views

Relation between low-rank approximation, nuclear norm of a matrix and Singular Value Decomposition

I'm reading the following paper https://arxiv.org/pdf/2005.10203.pdf which proposes improvements on robustness of large graphs to defend against adversarial attacks that are nothing but slight ...
1 vote
1 answer
24 views

Latent factors are the same in both decomposed matrices?

This question is in the context of recommendation systems. We can use matrix factorization techniques to decompose a user-product explicit/implicit matrix(R) into two matrices(U, P). Let's say R is a ...
  • 867
0 votes
0 answers
282 views

Why absolute value of eigenvalues are used in PCA or LDA?

In PCA and LDA techniques, eigenvectors with the $k$ largest eigenvalues give principal components. However, when selecting these eigenvalues, are they to be sorted by the absolute value (regardless ...
  • 1
0 votes
0 answers
51 views

Drawing samples from matrix normal

I have to generate $n \times m$ sample ($A$) from a matrix normal distribution, given two covariance matrices: $n \times n$ row covariance matrix (matrix $B$) (defines the covariance between the rows ...
  • 51
1 vote
0 answers
115 views

Are principal component analysis (PCA) and empirical orthogonal function (EOF) methods the same?

As far as I've seen, EOF is just PCA but instead of thinking about the data matrix X as (number of samples, number of features), you consider it as (number of time points, number of different spatial ...
0 votes
1 answer
121 views

Matrix Factorization and Linear Regression

Which matrix factorization algorithm is used in LinearRegression() function of scikit-learn?
1 vote
0 answers
134 views

Adding explicit user info to matrix factorization

In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this: $$ \hat{r}_{...
1 vote
1 answer
87 views

Understanding the decomposed matrices in Singular Value Decomposition [duplicate]

I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
  • 209
1 vote
1 answer
215 views

Why is there a discrepancy between the eigenvalues of the covariance matrix (PCA) and the eigenvalues of the kernel matrix (kernel PCA)?

I've done PCA on my data matrix $ \mathbf{X} $ which gives me i.a. the eigenvalues $ \lambda $ and eigenvectors $ v $ of the data covariance matrix $ C=\mathbf{X}^T \mathbf{X} $. I'm now extending my ...
  • 21
18 votes
1 answer
3k views

Relationship between Cholesky decomposition and matrix inversion?

I've been reviewing Gaussian Processes and, from what I can tell, there's some debate whether the "covariance matrix" (returned by the kernel), which needs to be inverted, should be done so ...
  • 1,728
1 vote
0 answers
165 views

Alternating least squares --- what are the limitations?

I am taking an introductory course to Machine Learning and we learned alternating least squares for recommender systems. I learned that this method has some advantages --- easy to parallelize, and ...
  • 11
2 votes
0 answers
40 views

Convex Optimization Algorithm for Additive Matrix Decomposition

Given an m $\times$ n matrix $Y = S + L + E$, where $S$ is a sparse matrix, $L$ is a low rank matrix and $E$ is a noise matrix, I want to recover $S$, $L$. One of the techniques given in Agarwal et al ...
1 vote
0 answers
143 views

Multivariate Normal Quadratic MGF: Eigendecomposition to Matrix form

If $X \sim \mathcal{N}(\mu, \Sigma)$ is a multivariate normal, then the quadratic $X^TAX$ has moment generating function $$M_{X^TAX}(t)= \frac{1}{\sqrt{\det(I - 2tA\Sigma)}}\exp\left(-\frac{1}{2}\mu'[...
1 vote
0 answers
42 views

Fastest way to find Leading singular value and vector (power iteration, rsvd etc)

I want to know the fastest way to find out the leading singular value and vector of a large rectangular matrix. I have seen 2 suggestions and have questions on both of them : Power Method : For this ...
2 votes
1 answer
148 views

How to explain the numerical discrepancy between FactoMineR::PCA() and the svd() in their output of the U matrix?

I am comparing the output of two functions in R to do Principal Component Analysis (PCA), the FactoMineR::PCA() and the ...
  • 1,085
3 votes
0 answers
219 views

Efficient computation of marginalized multivariate normal posterior distribution

In general,if we know that the marginal Gaussian distribution for some variable $\textbf{x}$ and a conditional Gaussian distribution for some $\textbf{y}|\textbf{x}$ of the forms: $$p(\textbf{x}) = \...
0 votes
0 answers
263 views

Covariance Matrix Decomposition - Data Decorrelation

So I recently found out about Mahalanobis distance. Given a r.v $x$ in N-dimensional space, an associated metric is defined by $$M(x) = \sqrt{(x-\mu)^T S^{-1}(x-\mu)}$$ where $\mu$ and $S$ are mean ...
1 vote
1 answer
57 views

How to compute the left singular eigenvector matrix (U) from the output of prcomp() for PCA in R?

I am examining the output of the prcomp() function in R for PCA in light of the singular value decomposition equation: $X = U \cdot \Sigma \cdot V^{T}$, where: $X$: ...
  • 1,085
1 vote
0 answers
30 views

How to find upper and lower bound

Let $\Sigma \in S_{++}^n$ be a symmteric positive definte matrix with all diagonal entries one. Let $U \in R^{n \times k_1}$, $W \in R^{n \times k_2}$, $\Lambda \in R^{k_1 \times k_1}$ and $T \in R^{...
  • 175
1 vote
0 answers
85 views

Kernel matrix decomposition

I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post: Nystroem Method for Kernel Approximation Here, a ...
  • 31
0 votes
0 answers
149 views

How to predict for test set when training a recommender by decomposing the utility matrix X=UV?

This probably sounds stupid but I don't get the workflow of building a recommending system by the utility matrix: X[i,j] = how much the ith user likes the jth object. For practical issues I refer to ...
1 vote
0 answers
49 views

SVD for a complex data matrix — what is the meaning of the columns of $V$?

I've read this wonderful explanation of SVD, where the writer mentions that the columns of $V$ are the principal directions (Summary, #1). Is this also true when the data matrix $X$ is complex? If I'm ...
  • 11
1 vote
1 answer
44 views

Convert the following expression w.r.t to the whole dataset instead of element of the dataset?

I am in the process of expressing the w in LSSVM with data points and constants. After I resolve the KKT conditions for the LSSVM I got $$w = \sum ^N _{i=1} \alpha_i x_i$$ Is it possible to convert ...
4 votes
1 answer
932 views

Prove that sample covariance matrix is positive definite [duplicate]

Consider the $p \times p$ sample covariance matrix: $$\mathbf{S} = \frac{1}{n-1} \cdot \mathbf{Y}_\mathbf{c}^\text{T} \mathbf{Y}_\mathbf{c} \quad \quad \quad \mathbf{Y}_\mathbf{c} = \mathbf{C} \mathbf{...
  • 211
3 votes
1 answer
545 views

Human intuition behind SVD in case of recommendation system

This does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the ...
  • 657
11 votes
3 answers
1k views

SVD : Why right singular matrix is written as transpose

The SVD is always written as, A = U Σ V_Transpose The question is, Why is the right singular matrix written as V_Transpose? I mean lets say, W = V_Transpose and then write SVD as A = U Σ W SVD Image ...
0 votes
1 answer
238 views

Penalized Canonical Correlation in R with PMA Module

I am trying to use sparse canonical correlation analysis as implemented in the R PMA package. I'm finding that the correlations output by the package seem slightly inconsistent with the ones you would ...
1 vote
0 answers
103 views

Symbolic Singular Value Decomposition? U,S,V as function of the elements of M [closed]

Suppose we want to compute the SVD of $\mathbf{M} = \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix}$ (...
1 vote
1 answer
38 views

Interpreting PCA results of first two components

I don't like the looks of my PCA graph here. PCA coordinates should be uncorrelated, yet the variance between the coordinates of the second component increases as the first component decreases. What's ...
  • 11
2 votes
0 answers
53 views

Use Matrix Factorization to predict probability of a recommendation system?

I have a dataset where I have a sparse utility matrix (user-product) with binary input: 1 if the user $i$ bought the product $j$, and 0 if it hasn't. However it has a different meaning on the test set,...
  • 21

1
2 3 4 5 6