Questions tagged [matrix-decomposition]
Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.
283
questions
0
votes
0
answers
5
views
State and Action spaces for MDP: Matrix as a state? Mixed action space? And more [closed]
I am pretty new to Reinforcement Learning (RL) and currently formulating the problem into a proper MDP. The goal is to restore the loads in a distribution system after an outage. The restoration is ...
1
vote
0
answers
17
views
General matrix decomposition downgrading algorithm for sampling
I would like to sample from a multivariate Gaussian distribution with covariance matrix $\Sigma - uu^T $, where $u$ is a vector and $\Sigma - uu^T $ is PSD. I have knowledge of a non-Cholesky ...
- 11
1
vote
1
answer
33
views
Latent Semantic Indexing vs. PCA
I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition.
Let's denote $X$ a $N \times D$ document-term matrix. The $D$ rows of $X$ ...
- 249
0
votes
0
answers
16
views
Understanding Leverage Score Sampling to get representative sample
I was reading about Leverage Score Sampling. If I am not wrong then what I know that Leverage Score Sampling help us to select representative sample. But I didn't understand how the whole process is ...
- 121
0
votes
0
answers
28
views
Correct NMF usage in context of recommender systems
I am trying to teach myself about the NMF models (in the context of recommender systems), and I have come across different suggestions on how to set up such a workflow, but I'm not sure if both are ...
- 101
1
vote
1
answer
35
views
Implementing eigen decomposition [closed]
Question
Please help understand why the eigen vectors do not match below. If there are misunderstandings or incorrect place, please correct too. It would be much appreciated.
Eigen decomposition
...
- 1,186
1
vote
0
answers
20
views
Eckart–Young–Mirsky theorem for $n \gg m$
It has been proven that the best reconstruction error in the $k$ rank matrix estimation problem in terms of Frobenius or $L2$ norm is given by the $k$-truncated SVD as shown here. I've read in ...
- 21
1
vote
0
answers
180
views
Why PCA is invariant under rotation
Lets say that we have a matrix of variables (the columns are variables and rows are the observations) called X whenre X = [x1, x2, ...., xp] where ...
- 133
0
votes
0
answers
18
views
Generate dataset that adhere to a given correlation matrix [duplicate]
I'm newbie in statistics and this question might be naive so kindly excuse me in advance. I have a correlation matrix of 40 features and I want to generate a dataset of hundreds of observations that ...
- 101
1
vote
0
answers
31
views
Decomposition analysis for data between zero and one
I want to analyze latent components of data that has values between zero and one (including zero and one).
In detail, the data structure is n x m and I'm looking to find the r underlying components.
...
- 31
0
votes
0
answers
42
views
How to find a decomposition of multivariate X along which y varies the most?
I'm looking for an existing algorithm which carries out the task shown in the title.
My use-case in other words: I have a set of continuous independent variables (X) and a continuous dependent ...
- 123
1
vote
0
answers
22
views
quasi-PCA reconstruction of the matrix by orthogonal basis
Let's say I have a "data" matrix $X$ of $N$ rows and $p$ cols with $N \gg p$. Now PCA with $L$ components can be formulated as
$$
X_L = \underset{Y:rank(Y) = L}{\text{argmin}} ||X- Y||^2_F, ...
0
votes
0
answers
35
views
The relationship between eigenvalues of a covariance matrix and the variances of the same data matrix after using eigenvectors as bases
Suppose we have a data matrix $\mathbf{X}\in \mathbb{R}^{M\times N}$ with $M$ features, $N$ samples and zero means ($M \lt N$). The covariance matrix of $\mathbf{X}$ is $\mathbf{C_x}=\frac{1}{N}\...
- 1
0
votes
0
answers
84
views
Decomposing Distance Matrix D for approximating Original Matrix A
Let's say we have a matrix $A \in R^{n \times d}$ where n is the number of elements and d is the dimension size. And we calculate the pairwise distances between each elements; say cosine for instance ...
- 103
0
votes
0
answers
100
views
Find the principal component and the proportion of the total population variance explained by each when the variance covariance matrix is given
I can understand the part where we have to find the principal component from the variane covariance matrix- find eigen values, make eigen vector and normalise. The principal component would be ...
- 105
1
vote
1
answer
103
views
Matrix dimensions in Linear Algebra vs Time series Analysis
I am confused or may misunderstand the dimensions of a Matrix when I was reading about time series analysis.
From what I understand in linear Algebra, if we have a Matrix $A \in \mathbf{R}^{m*n}$, ...
- 13
0
votes
0
answers
346
views
Funk SVD for binary data - product like or dislike
Assume the following situation:
you have a user-item sparse matrix. However, instead of the usual 1 to 5 rating scale, items can only receive a positive (1) or negative (-1) feedback.
Thus, the matrix ...
- 143
1
vote
0
answers
12
views
CP Tensor Decomposition and Correlating Sample Magnitudes with Variables of Interest
I am learning about tensor decomposition, specifically CP, and am trying to understand if I can use it for my research.
To give a bit more detail, I have brain imaging data from 10 participants, with ...
- 325
0
votes
0
answers
78
views
Constrained Matrix Decomposition
I am working on a structural vector autoregression that requires imposing constraints on a matrix factorization.
In particular, I have an N-dimensional positive definite matrix $\Sigma$ that I need to ...
- 2,197
1
vote
1
answer
66
views
How to decompose a random walk (array) into its Markov Chain transition matrix?
The algorithm, PageRank, receives a Markov Chain transition matrix (page links from one to another.) Either by random walk, or more efficiently, eigenvectors, the stationary distribution of the Markov ...
- 2,113
1
vote
1
answer
40
views
Is the first independent component of independent component analysis always important?
I was looking at a neuroscience paper that used ICA to reduce dimensionality of calcium signaling profiles in 20 randomly selected neurons of a zebrafish brain.
I presume that in Figure 2, ICA was ...
- 11
1
vote
0
answers
25
views
Relation between low-rank approximation, nuclear norm of a matrix and Singular Value Decomposition
I'm reading the following paper https://arxiv.org/pdf/2005.10203.pdf which proposes improvements on robustness of large graphs to defend against adversarial attacks that are nothing but slight ...
- 537
1
vote
1
answer
41
views
Latent factors are the same in both decomposed matrices?
This question is in the context of recommendation systems. We can use matrix factorization techniques to decompose a user-product explicit/implicit matrix(R) into two matrices(U, P). Let's say R is a ...
- 887
0
votes
0
answers
481
views
Why absolute value of eigenvalues are used in PCA or LDA?
In PCA and LDA techniques, eigenvectors with the $k$ largest eigenvalues give principal components. However, when selecting these eigenvalues, are they to be sorted by the absolute value (regardless ...
- 1
0
votes
0
answers
60
views
Drawing samples from matrix normal
I have to generate $n \times m$ sample ($A$) from a matrix normal distribution, given two covariance matrices:
$n \times n$ row covariance matrix (matrix $B$) (defines the covariance between the rows ...
- 51
1
vote
0
answers
198
views
Are principal component analysis (PCA) and empirical orthogonal function (EOF) methods the same?
As far as I've seen, EOF is just PCA but instead of thinking about the data matrix X as (number of samples, number of features), you consider it as (number of time points, number of different spatial ...
0
votes
1
answer
139
views
Matrix Factorization and Linear Regression
Which matrix factorization algorithm is used in LinearRegression() function of scikit-learn?
- 21
1
vote
0
answers
163
views
Adding explicit user info to matrix factorization
In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this:
$$
\hat{r}_{...
1
vote
1
answer
126
views
Understanding the decomposed matrices in Singular Value Decomposition [duplicate]
I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
- 249
1
vote
1
answer
300
views
Why is there a discrepancy between the eigenvalues of the covariance matrix (PCA) and the eigenvalues of the kernel matrix (kernel PCA)?
I've done PCA on my data matrix $ \mathbf{X} $ which gives me i.a. the eigenvalues $ \lambda $ and eigenvectors $ v $ of the data covariance matrix $ C=\mathbf{X}^T \mathbf{X} $. I'm now extending my ...
- 21
19
votes
1
answer
5k
views
Relationship between Cholesky decomposition and matrix inversion?
I've been reviewing Gaussian Processes and, from what I can tell, there's some debate whether the "covariance matrix" (returned by the kernel), which needs to be inverted, should be done so ...
- 2,113
1
vote
0
answers
206
views
Alternating least squares --- what are the limitations?
I am taking an introductory course to Machine Learning and we learned alternating least squares for recommender systems. I learned that this method has some advantages --- easy to parallelize, and ...
- 11
2
votes
0
answers
41
views
Convex Optimization Algorithm for Additive Matrix Decomposition
Given an m $\times$ n matrix $Y = S + L + E$, where $S$ is a sparse matrix, $L$ is a low rank matrix and $E$ is a noise matrix, I want to recover $S$, $L$. One of the techniques given in Agarwal et al ...
1
vote
0
answers
177
views
Multivariate Normal Quadratic MGF: Eigendecomposition to Matrix form
If $X \sim \mathcal{N}(\mu, \Sigma)$ is a multivariate normal, then the quadratic $X^TAX$ has moment generating function
$$M_{X^TAX}(t)= \frac{1}{\sqrt{\det(I - 2tA\Sigma)}}\exp\left(-\frac{1}{2}\mu'[...
- 111
1
vote
0
answers
53
views
Fastest way to find Leading singular value and vector (power iteration, rsvd etc)
I want to know the fastest way to find out the leading singular value and vector of a large rectangular matrix. I have seen 2 suggestions and have questions on both of them :
Power Method : For this ...
2
votes
1
answer
201
views
How to explain the numerical discrepancy between FactoMineR::PCA() and the svd() in their output of the U matrix?
I am comparing the output of two functions in R to do Principal Component Analysis (PCA), the FactoMineR::PCA() and the ...
- 1,085
3
votes
0
answers
305
views
Efficient computation of marginalized multivariate normal posterior distribution
In general,if we know that the marginal Gaussian distribution for some variable $\textbf{x}$ and a conditional Gaussian distribution for some $\textbf{y}|\textbf{x}$ of the forms:
$$p(\textbf{x}) = \...
- 51
0
votes
0
answers
355
views
Covariance Matrix Decomposition - Data Decorrelation
So I recently found out about Mahalanobis distance. Given a r.v $x$ in N-dimensional space, an associated metric is defined by
$$M(x) = \sqrt{(x-\mu)^T S^{-1}(x-\mu)}$$
where $\mu$ and $S$ are mean ...
- 1,749
1
vote
1
answer
84
views
How to compute the left singular eigenvector matrix (U) from the output of prcomp() for PCA in R?
I am examining the output of the prcomp() function in R for PCA in light of the singular value decomposition equation:
$X = U \cdot \Sigma \cdot V^{T}$, where:
$X$: ...
- 1,085
1
vote
0
answers
41
views
How to find upper and lower bound
Let $\Sigma \in S_{++}^n$ be a symmteric positive definte matrix with all diagonal entries one. Let $U \in R^{n \times k_1}$, $W \in R^{n \times k_2}$, $\Lambda \in R^{k_1 \times k_1}$ and $T \in R^{...
- 205
1
vote
0
answers
138
views
Kernel matrix decomposition
I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post:
Nystroem Method for Kernel Approximation
Here, a ...
- 31
0
votes
0
answers
228
views
How to predict for test set when training a recommender by decomposing the utility matrix X=UV?
This probably sounds stupid but I don't get the workflow of building a recommending system by the utility matrix: X[i,j] = how much the ith user likes the jth object. For practical issues I refer to ...
- 143
1
vote
0
answers
64
views
SVD for a complex data matrix — what is the meaning of the columns of $V$?
I've read this wonderful explanation of SVD, where the writer mentions that the columns of $V$ are the principal directions (Summary, #1). Is this also true when the data matrix $X$ is complex?
If I'm ...
- 11
1
vote
1
answer
46
views
Convert the following expression w.r.t to the whole dataset instead of element of the dataset?
I am in the process of expressing the w in LSSVM with data points and constants. After I resolve the KKT conditions for the LSSVM I got
$$w = \sum ^N _{i=1} \alpha_i x_i$$
Is it possible to convert ...
- 55
4
votes
1
answer
1k
views
Prove that sample covariance matrix is positive definite [duplicate]
Consider the $p \times p$ sample covariance matrix:
$$\mathbf{S} = \frac{1}{n-1} \cdot \mathbf{Y}_\mathbf{c}^\text{T} \mathbf{Y}_\mathbf{c}
\quad \quad \quad
\mathbf{Y}_\mathbf{c} = \mathbf{C} \mathbf{...
- 211
3
votes
1
answer
675
views
Human intuition behind SVD in case of recommendation system
This does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the ...
- 667
11
votes
3
answers
1k
views
SVD : Why right singular matrix is written as transpose
The SVD is always written as,
A = U Σ V_Transpose
The question is,
Why is the right singular matrix written as V_Transpose?
I mean lets say,
W = V_Transpose
and then write SVD as
A = U Σ W
SVD Image ...
0
votes
1
answer
283
views
Penalized Canonical Correlation in R with PMA Module
I am trying to use sparse canonical correlation analysis as implemented in the R PMA package. I'm finding that the correlations output by the package seem slightly inconsistent with the ones you would ...
1
vote
0
answers
130
views
Symbolic Singular Value Decomposition? U,S,V as function of the elements of M [closed]
Suppose we want to compute the SVD of $\mathbf{M} = \begin{bmatrix}
m_{11} & m_{12} & m_{13} \\
m_{21} & m_{22} & m_{23} \\
m_{31} & m_{32} & m_{33}
\end{bmatrix}$ (...
- 11
1
vote
1
answer
45
views
Interpreting PCA results of first two components
I don't like the looks of my PCA graph here. PCA coordinates should be uncorrelated, yet the variance between the coordinates of the second component increases as the first component decreases. What's ...
- 11