Questions tagged [matrix-decomposition]
Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.
295 questions
0
votes
0
answers
26
views
Decomposing correlation matrices into vectors which have pairwise bivariate normal entries to apply test of significance?
This question originally was posted at MathOverflow, but I deleted it there, since it might be a better fit for here:
A semantic space is a finite set $X$ with a symmetric, positive semi-definite ...
- 129
0
votes
0
answers
20
views
Fast covariance evaluation for sparse and large-scale non-linear least square problems
A non-linear least squares problem can be formulated as:
$$
\hat{x} = \underset{x}{\mathrm{argmin}}\, \frac{1}{2} ||\varepsilon(x)||^2
$$
where $x$ is the parameters of the problem, and $\varepsilon(x)...
- 556
0
votes
1
answer
26
views
Sequential sum of squares with svd
I am studying some methods to determine the coefficients of a linear regression and I am wondering how to find the sequential sum of squares, or the second column of the ANOVA table which shows ...
- 281
3
votes
1
answer
98
views
The Math Behind the Conditional Probability of a Probabilistic PCA
I am trying to understand how to calculate the conditional distribution of probabilistic principal component analysis. This is explained in the book "Pattern Recognition and Machine Learning"...
- 75
0
votes
1
answer
30
views
Decomposition of VAR(1) coefficient matrix
Consider the VAR(1) process $X_t = \Phi X_{t-1} + \epsilon_t.$
Is there a generally accepted decomposition for the coefficient matrix $\Phi$ that would decrease the degrees of freedom?
My initial ...
- 81
2
votes
1
answer
104
views
Passing a cholesky decomposition for a matrix with constrained variances to an objective function
I am trying to optimize an objective function $L(\theta)$ in which some parameters that I aim to recover belong to a covariance matrix, $\Sigma$. $\Sigma$ has a unique structure, which includes ones ...
- 33
0
votes
2
answers
77
views
The Impact of Vector Magnitudes in Recommendation Systems Matrix Factorization Models
I'm currently exploring latent factor models in recommendation systems, specifically focusing on the interaction between vector magnitudes and the angles between these vectors. While it's clear that ...
- 77
1
vote
1
answer
19
views
Efficient construction of correlation matrix—serial correlation
Given $\rho$, is there a way to efficiently construct this matrix (i.e., as a product of matrices, rather than using a for loop)?
$$
\Sigma =
\begin{pmatrix}
1 & \rho & \rho^2 &\cdots &...
4
votes
0
answers
181
views
Fast Cholesky decomposition of a Toepllitz matrix via embedding in a circulant & fft
As I understand it, the Cholesky decomposition of a Toeplitz matrix can be computed more efficiently by first embedding it in a circulant matrix then using FFT, but I'm having trouble finding any ...
- 14k
2
votes
0
answers
14
views
Is spiked tensor decomposition a special case of INDSCAL decomposition?
I understand that "Spiked" often refers to the presence of a dominant component (or a few dominant components) in a tensor decomposition. Spiked tensor decomposition is applied to multi-way ...
- 95
1
vote
1
answer
99
views
Method of least squares, first order condition and QR decomposition
When you use the method of least squares you estimate the parameters in the following way:
$$\min_{\mathbf{b}} (\mathbf{y} - \mathbf{X}\mathbf{b})^T(\mathbf{y} - \mathbf{X}\mathbf{b})$$
Where $\mathbf{...
- 111
0
votes
0
answers
31
views
Matrix Factorization with SGD gets na results
I'm trying to implement MF with SGD to my sample data following thru https://nbviewer.org/github/albertauyeung/matrix-factorization-in-python/blob/master/mf.ipynb.
And, it's hard to figure out why the ...
- 1
0
votes
0
answers
14
views
Sensitivity analysis of a fuzzy cognitive model
I have a fuzzy cognitive model of inter-organizational collaboration that is represented by a 27x27 matrix. I want to analyze the effects of individual variables. I've read I can do this best with a ...
3
votes
1
answer
121
views
Properties of the generalized centering matrix [closed]
Let $L_{\boldsymbol{\Delta}}\in \mathbb{R}^{M \times M}$ be the generalized centering matrix given by:
$L_{\boldsymbol{\Delta}} = \boldsymbol{\Delta} - \frac{1}{\text{tr}(\boldsymbol{\Delta})} \...
- 33
1
vote
0
answers
24
views
Prediction of Multiple Linear Regression With Constant
Let $X$ be a matrix with $n$ rows and $d$ columns. We know that there exists matrices $U, S, V$, with $U$ of dimensions $(n, d)$, $S$ of dimensions $(d, d)$ and $V$ of dimensions $(d, d)$, which form ...
- 21
2
votes
0
answers
71
views
Understanding diagonal rescaling in multiplicative update rules for NMF
SUMMARY
How does the diagonal rescaling fit into the derivation of a multiplicative update rule for non-negative matrix factorization (NMF)?
DESCRIPTION
The NMF problem aims to find non-negative ...
- 21
1
vote
0
answers
33
views
General matrix decomposition downgrading algorithm for sampling
I would like to sample from a multivariate Gaussian distribution with covariance matrix $\Sigma - uu^T $, where $u$ is a vector and $\Sigma - uu^T $ is PSD. I have knowledge of a non-Cholesky ...
- 11
2
votes
1
answer
86
views
Latent Semantic Indexing vs. PCA
I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition.
Let's denote $X$ a $N \times D$ document-term matrix. The $D$ rows of $X$ ...
- 319
2
votes
1
answer
1k
views
Understanding Leverage Score Sampling to get representative sample
I was reading about Leverage Score Sampling. If I am not wrong then what I know that Leverage Score Sampling help us to select representative sample. But I didn't understand how the whole process is ...
- 141
1
vote
1
answer
70
views
Implementing eigen decomposition [closed]
Question
Please help understand why the eigen vectors do not match below. If there are misunderstandings or incorrect place, please correct too. It would be much appreciated.
Eigen decomposition
...
- 1,568
1
vote
0
answers
24
views
Eckart–Young–Mirsky theorem for $n \gg m$
It has been proven that the best reconstruction error in the $k$ rank matrix estimation problem in terms of Frobenius or $L2$ norm is given by the $k$-truncated SVD as shown here. I've read in ...
- 121
1
vote
0
answers
685
views
Why PCA is invariant under rotation
Lets say that we have a matrix of variables (the columns are variables and rows are the observations) called X whenre X = [x1, x2, ...., xp] where ...
- 143
1
vote
0
answers
33
views
Decomposition analysis for data between zero and one
I want to analyze latent components of data that has values between zero and one (including zero and one).
In detail, the data structure is n x m and I'm looking to find the r underlying components.
...
- 31
0
votes
0
answers
49
views
How to find a decomposition of multivariate X along which y varies the most?
I'm looking for an existing algorithm which carries out the task shown in the title.
My use-case in other words: I have a set of continuous independent variables (X) and a continuous dependent ...
- 123
1
vote
0
answers
23
views
quasi-PCA reconstruction of the matrix by orthogonal basis
Let's say I have a "data" matrix $X$ of $N$ rows and $p$ cols with $N \gg p$. Now PCA with $L$ components can be formulated as
$$
X_L = \underset{Y:rank(Y) = L}{\text{argmin}} ||X- Y||^2_F, ...
0
votes
0
answers
78
views
The relationship between eigenvalues of a covariance matrix and the variances of the same data matrix after using eigenvectors as bases
Suppose we have a data matrix $\mathbf{X}\in \mathbb{R}^{M\times N}$ with $M$ features, $N$ samples and zero means ($M \lt N$). The covariance matrix of $\mathbf{X}$ is $\mathbf{C_x}=\frac{1}{N}\...
- 101
0
votes
0
answers
158
views
Decomposing Distance Matrix D for approximating Original Matrix A
Let's say we have a matrix $A \in R^{n \times d}$ where n is the number of elements and d is the dimension size. And we calculate the pairwise distances between each elements; say cosine for instance ...
- 103
2
votes
1
answer
261
views
Matrix dimensions in Linear Algebra vs Time series Analysis
I am confused or may misunderstand the dimensions of a Matrix when I was reading about time series analysis.
From what I understand in linear Algebra, if we have a Matrix $A \in \mathbf{R}^{m*n}$, ...
- 23
0
votes
0
answers
555
views
Funk SVD for binary data - product like or dislike
Assume the following situation:
you have a user-item sparse matrix. However, instead of the usual 1 to 5 rating scale, items can only receive a positive (1) or negative (-1) feedback.
Thus, the matrix ...
- 143
1
vote
0
answers
16
views
CP Tensor Decomposition and Correlating Sample Magnitudes with Variables of Interest
I am learning about tensor decomposition, specifically CP, and am trying to understand if I can use it for my research.
To give a bit more detail, I have brain imaging data from 10 participants, with ...
- 335
0
votes
0
answers
149
views
Constrained Matrix Decomposition
I am working on a structural vector autoregression that requires imposing constraints on a matrix factorization.
In particular, I have an N-dimensional positive definite matrix $\Sigma$ that I need to ...
- 2,297
1
vote
1
answer
108
views
How to decompose a random walk (array) into its Markov Chain transition matrix?
The algorithm, PageRank, receives a Markov Chain transition matrix (page links from one to another.) Either by random walk, or more efficiently, eigenvectors, the stationary distribution of the Markov ...
- 3,520
2
votes
1
answer
69
views
Is the first independent component of independent component analysis always important?
I was looking at a neuroscience paper that used ICA to reduce dimensionality of calcium signaling profiles in 20 randomly selected neurons of a zebrafish brain.
I presume that in Figure 2, ICA was ...
- 21
1
vote
0
answers
30
views
Relation between low-rank approximation, nuclear norm of a matrix and Singular Value Decomposition
I'm reading the following paper https://arxiv.org/pdf/2005.10203.pdf which proposes improvements on robustness of large graphs to defend against adversarial attacks that are nothing but slight ...
- 659
2
votes
1
answer
68
views
Latent factors are the same in both decomposed matrices?
This question is in the context of recommendation systems. We can use matrix factorization techniques to decompose a user-product explicit/implicit matrix(R) into two matrices(U, P). Let's say R is a ...
- 1,027
1
vote
0
answers
937
views
Why absolute value of eigenvalues are used in PCA or LDA?
In PCA and LDA techniques, eigenvectors with the $k$ largest eigenvalues give principal components. However, when selecting these eigenvalues, are they to be sorted by the absolute value (regardless ...
- 11
0
votes
0
answers
73
views
Drawing samples from matrix normal
I have to generate $n \times m$ sample ($A$) from a matrix normal distribution, given two covariance matrices:
$n \times n$ row covariance matrix (matrix $B$) (defines the covariance between the rows ...
- 51
1
vote
0
answers
334
views
Are principal component analysis (PCA) and empirical orthogonal function (EOF) methods the same?
As far as I've seen, EOF is just PCA but instead of thinking about the data matrix X as (number of samples, number of features), you consider it as (number of time points, number of different spatial ...
0
votes
1
answer
169
views
Matrix Factorization and Linear Regression
Which matrix factorization algorithm is used in LinearRegression() function of scikit-learn?
- 41
2
votes
0
answers
244
views
Adding explicit user info to matrix factorization
In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this:
$$
\hat{r}_{...
1
vote
1
answer
232
views
Understanding the decomposed matrices in Singular Value Decomposition [duplicate]
I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
- 319
1
vote
1
answer
535
views
Why is there a discrepancy between the eigenvalues of the covariance matrix (PCA) and the eigenvalues of the kernel matrix (kernel PCA)?
I've done PCA on my data matrix $ \mathbf{X} $ which gives me i.a. the eigenvalues $ \lambda $ and eigenvectors $ v $ of the data covariance matrix $ C=\mathbf{X}^T \mathbf{X} $. I'm now extending my ...
- 21
22
votes
1
answer
8k
views
Relationship between Cholesky decomposition and matrix inversion?
I've been reviewing Gaussian Processes and, from what I can tell, there's some debate whether the "covariance matrix" (returned by the kernel), which needs to be inverted, should be done so ...
- 3,520
1
vote
0
answers
270
views
Alternating least squares --- what are the limitations?
I am taking an introductory course to Machine Learning and we learned alternating least squares for recommender systems. I learned that this method has some advantages --- easy to parallelize, and ...
- 11
2
votes
0
answers
49
views
Convex Optimization Algorithm for Additive Matrix Decomposition
Given an m $\times$ n matrix $Y = S + L + E$, where $S$ is a sparse matrix, $L$ is a low rank matrix and $E$ is a noise matrix, I want to recover $S$, $L$. One of the techniques given in Agarwal et al ...
1
vote
0
answers
283
views
Multivariate Normal Quadratic MGF: Eigendecomposition to Matrix form
If $X \sim \mathcal{N}(\mu, \Sigma)$ is a multivariate normal, then the quadratic $X^TAX$ has moment generating function
$$M_{X^TAX}(t)= \frac{1}{\sqrt{\det(I - 2tA\Sigma)}}\exp\left(-\frac{1}{2}\mu'[...
- 111
1
vote
0
answers
107
views
Fastest way to find Leading singular value and vector (power iteration, rsvd etc)
I want to know the fastest way to find out the leading singular value and vector of a large rectangular matrix. I have seen 2 suggestions and have questions on both of them :
Power Method : For this ...
3
votes
1
answer
305
views
How to explain the numerical discrepancy between FactoMineR::PCA() and the svd() in their output of the U matrix?
I am comparing the output of two functions in R to do Principal Component Analysis (PCA), the FactoMineR::PCA() and the ...
- 1,147
3
votes
0
answers
427
views
Efficient computation of marginalized multivariate normal posterior distribution
In general,if we know that the marginal Gaussian distribution for some variable $\textbf{x}$ and a conditional Gaussian distribution for some $\textbf{y}|\textbf{x}$ of the forms:
$$p(\textbf{x}) = \...
- 51
1
vote
0
answers
573
views
Covariance Matrix Decomposition - Data Decorrelation
So I recently found out about Mahalanobis distance. Given a r.v $x$ in N-dimensional space, an associated metric is defined by
$$M(x) = \sqrt{(x-\mu)^T S^{-1}(x-\mu)}$$
where $\mu$ and $S$ are mean ...
- 2,131