Questions tagged [matrix-decomposition]

Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.

Filter by
Sorted by
Tagged with
0 votes
1 answer
26 views

Decomposition of VAR(1) coefficient matrix

Consider the VAR(1) process $X_t = \Phi X_{t-1} + \epsilon_t.$ Is there a generally accepted decomposition for the coefficient matrix $\Phi$ that would decrease the degrees of freedom? My initial ...
Ville's user avatar
  • 71
2 votes
1 answer
58 views

Passing a cholesky decomposition for a matrix with constrained variances to an objective function

I am trying to optimize an objective function $L(\theta)$ in which some parameters that I aim to recover belong to a covariance matrix, $\Sigma$. $\Sigma$ has a unique structure, which includes ones ...
EB727's user avatar
  • 33
0 votes
2 answers
72 views

The Impact of Vector Magnitudes in Recommendation Systems Matrix Factorization Models

I'm currently exploring latent factor models in recommendation systems, specifically focusing on the interaction between vector magnitudes and the angles between these vectors. While it's clear that ...
Amit S's user avatar
  • 57
1 vote
1 answer
16 views

Efficient construction of correlation matrix—serial correlation

Given $\rho$, is there a way to efficiently construct this matrix (i.e., as a product of matrices, rather than using a for loop)? $$ \Sigma = \begin{pmatrix} 1 & \rho & \rho^2 &\cdots &...
veloskaraptor's user avatar
4 votes
0 answers
106 views

Fast Cholesky decomposition of a Toepllitz matrix via embedding in a circulant & fft

As I understand it, the Cholesky decomposition of a Toeplitz matrix can be computed more efficiently by first embedding it in a circulant matrix then using FFT, but I'm having trouble finding any ...
Mike Lawrence's user avatar
2 votes
0 answers
10 views

Is spiked tensor decomposition a special case of INDSCAL decomposition?

I understand that "Spiked" often refers to the presence of a dominant component (or a few dominant components) in a tensor decomposition. Spiked tensor decomposition is applied to multi-way ...
Omar Shehab's user avatar
1 vote
1 answer
93 views

Method of least squares, first order condition and QR decomposition

When you use the method of least squares you estimate the parameters in the following way: $$\min_{\mathbf{b}} (\mathbf{y} - \mathbf{X}\mathbf{b})^T(\mathbf{y} - \mathbf{X}\mathbf{b})$$ Where $\mathbf{...
luifrancgom's user avatar
0 votes
0 answers
12 views

How can a linear autoencoder with $h=1$ hidden unit reconstruct any rank 1 matrix?

I've had this as a homework problem as a true or false type of question and I'm trying to wrap my head around why this is true. Is the reason simply represent each datapoint as a scaled version of a ...
Oliver's user avatar
  • 290
0 votes
0 answers
9 views

Does it makes sense to prune filters in convolutional layer when rank of filters is smaller than number of filters?

Let’s consider a convolutional layer, taking filters (matrices used in convolution), vectorizing them, putting them into matrix $A$ and computing it's rank. Does it makes sense to remove $|filters|-...
quester's user avatar
  • 498
0 votes
0 answers
25 views

Matrix Factorization with SGD gets na results

I'm trying to implement MF with SGD to my sample data following thru https://nbviewer.org/github/albertauyeung/matrix-factorization-in-python/blob/master/mf.ipynb. And, it's hard to figure out why the ...
Donsuke's user avatar
0 votes
0 answers
18 views

Is it possible to apply matrix decomposition to a vector, injecting additional information to UV decomposition?

As I am reading about recommender systems in Machine Learning, UV decomposition caught my eye (click for an explanation or see below). So I have two questions: Question 1: what are the drawbacks of ...
Jironymo's user avatar
0 votes
0 answers
13 views

Sensitivity analysis of a fuzzy cognitive model

I have a fuzzy cognitive model of inter-organizational collaboration that is represented by a 27x27 matrix. I want to analyze the effects of individual variables. I've read I can do this best with a ...
Peter de KAM's user avatar
3 votes
1 answer
100 views

Properties of the generalized centering matrix [closed]

Let $L_{\boldsymbol{\Delta}}\in \mathbb{R}^{M \times M}$ be the generalized centering matrix given by: $L_{\boldsymbol{\Delta}} = \boldsymbol{\Delta} - \frac{1}{\text{tr}(\boldsymbol{\Delta})} \...
Mathieu's user avatar
  • 33
1 vote
0 answers
22 views

Prediction of Multiple Linear Regression With Constant

Let $X$ be a matrix with $n$ rows and $d$ columns. We know that there exists matrices $U, S, V$, with $U$ of dimensions $(n, d)$, $S$ of dimensions $(d, d)$ and $V$ of dimensions $(d, d)$, which form ...
user35083's user avatar
2 votes
0 answers
47 views

Understanding diagonal rescaling in multiplicative update rules for NMF

SUMMARY How does the diagonal rescaling fit into the derivation of a multiplicative update rule for non-negative matrix factorization (NMF)? DESCRIPTION The NMF problem aims to find non-negative ...
scho's user avatar
  • 21
1 vote
0 answers
29 views

General matrix decomposition downgrading algorithm for sampling

I would like to sample from a multivariate Gaussian distribution with covariance matrix $\Sigma - uu^T $, where $u$ is a vector and $\Sigma - uu^T $ is PSD. I have knowledge of a non-Cholesky ...
Noam Elata's user avatar
2 votes
1 answer
66 views

Latent Semantic Indexing vs. PCA

I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition. Let's denote $X$ a $N \times D$ document-term matrix. The $D$ rows of $X$ ...
Victor M's user avatar
  • 309
1 vote
1 answer
773 views

Understanding Leverage Score Sampling to get representative sample

I was reading about Leverage Score Sampling. If I am not wrong then what I know that Leverage Score Sampling help us to select representative sample. But I didn't understand how the whole process is ...
falamiw's user avatar
  • 131
1 vote
1 answer
58 views

Implementing eigen decomposition [closed]

Question Please help understand why the eigen vectors do not match below. If there are misunderstandings or incorrect place, please correct too. It would be much appreciated. Eigen decomposition ...
mon's user avatar
  • 1,468
1 vote
0 answers
22 views

Eckart–Young–Mirsky theorem for $n \gg m$

It has been proven that the best reconstruction error in the $k$ rank matrix estimation problem in terms of Frobenius or $L2$ norm is given by the $k$-truncated SVD as shown here. I've read in ...
jonithani123's user avatar
1 vote
0 answers
529 views

Why PCA is invariant under rotation

Lets say that we have a matrix of variables (the columns are variables and rows are the observations) called X whenre X = [x1, x2, ...., xp] where ...
Amir Jalilifard's user avatar
1 vote
0 answers
33 views

Decomposition analysis for data between zero and one

I want to analyze latent components of data that has values between zero and one (including zero and one). In detail, the data structure is n x m and I'm looking to find the r underlying components. ...
gnm's user avatar
  • 31
0 votes
0 answers
47 views

How to find a decomposition of multivariate X along which y varies the most?

I'm looking for an existing algorithm which carries out the task shown in the title. My use-case in other words: I have a set of continuous independent variables (X) and a continuous dependent ...
Haliaetus's user avatar
  • 123
1 vote
0 answers
23 views

quasi-PCA reconstruction of the matrix by orthogonal basis

Let's say I have a "data" matrix $X$ of $N$ rows and $p$ cols with $N \gg p$. Now PCA with $L$ components can be formulated as $$ X_L = \underset{Y:rank(Y) = L}{\text{argmin}} ||X- Y||^2_F, ...
Vladimir Kirilin's user avatar
0 votes
0 answers
72 views

The relationship between eigenvalues of a covariance matrix and the variances of the same data matrix after using eigenvectors as bases

Suppose we have a data matrix $\mathbf{X}\in \mathbb{R}^{M\times N}$ with $M$ features, $N$ samples and zero means ($M \lt N$). The covariance matrix of $\mathbf{X}$ is $\mathbf{C_x}=\frac{1}{N}\...
tyrela's user avatar
  • 101
0 votes
0 answers
140 views

Decomposing Distance Matrix D for approximating Original Matrix A

Let's say we have a matrix $A \in R^{n \times d}$ where n is the number of elements and d is the dimension size. And we calculate the pairwise distances between each elements; say cosine for instance ...
Kadir Gunel's user avatar
1 vote
1 answer
190 views

Matrix dimensions in Linear Algebra vs Time series Analysis

I am confused or may misunderstand the dimensions of a Matrix when I was reading about time series analysis. From what I understand in linear Algebra, if we have a Matrix $A \in \mathbf{R}^{m*n}$, ...
A Sam's user avatar
  • 13
0 votes
0 answers
519 views

Funk SVD for binary data - product like or dislike

Assume the following situation: you have a user-item sparse matrix. However, instead of the usual 1 to 5 rating scale, items can only receive a positive (1) or negative (-1) feedback. Thus, the matrix ...
wtfzambo's user avatar
  • 143
1 vote
0 answers
16 views

CP Tensor Decomposition and Correlating Sample Magnitudes with Variables of Interest

I am learning about tensor decomposition, specifically CP, and am trying to understand if I can use it for my research. To give a bit more detail, I have brain imaging data from 10 participants, with ...
pdhami's user avatar
  • 335
0 votes
0 answers
128 views

Constrained Matrix Decomposition

I am working on a structural vector autoregression that requires imposing constraints on a matrix factorization. In particular, I have an N-dimensional positive definite matrix $\Sigma$ that I need to ...
John's user avatar
  • 2,287
1 vote
1 answer
92 views

How to decompose a random walk (array) into its Markov Chain transition matrix?

The algorithm, PageRank, receives a Markov Chain transition matrix (page links from one to another.) Either by random walk, or more efficiently, eigenvectors, the stationary distribution of the Markov ...
jbuddy_13's user avatar
  • 3,020
1 vote
1 answer
53 views

Is the first independent component of independent component analysis always important?

I was looking at a neuroscience paper that used ICA to reduce dimensionality of calcium signaling profiles in 20 randomly selected neurons of a zebrafish brain. I presume that in Figure 2, ICA was ...
John's user avatar
  • 11
1 vote
0 answers
26 views

Relation between low-rank approximation, nuclear norm of a matrix and Singular Value Decomposition

I'm reading the following paper https://arxiv.org/pdf/2005.10203.pdf which proposes improvements on robustness of large graphs to defend against adversarial attacks that are nothing but slight ...
James Arten's user avatar
1 vote
1 answer
64 views

Latent factors are the same in both decomposed matrices?

This question is in the context of recommendation systems. We can use matrix factorization techniques to decompose a user-product explicit/implicit matrix(R) into two matrices(U, P). Let's say R is a ...
etang's user avatar
  • 977
1 vote
0 answers
815 views

Why absolute value of eigenvalues are used in PCA or LDA?

In PCA and LDA techniques, eigenvectors with the $k$ largest eigenvalues give principal components. However, when selecting these eigenvalues, are they to be sorted by the absolute value (regardless ...
Ananda's user avatar
  • 11
0 votes
0 answers
68 views

Drawing samples from matrix normal

I have to generate $n \times m$ sample ($A$) from a matrix normal distribution, given two covariance matrices: $n \times n$ row covariance matrix (matrix $B$) (defines the covariance between the rows ...
borodor's user avatar
  • 51
1 vote
0 answers
308 views

Are principal component analysis (PCA) and empirical orthogonal function (EOF) methods the same?

As far as I've seen, EOF is just PCA but instead of thinking about the data matrix X as (number of samples, number of features), you consider it as (number of time points, number of different spatial ...
Rogelio Lozano's user avatar
0 votes
1 answer
160 views

Matrix Factorization and Linear Regression

Which matrix factorization algorithm is used in LinearRegression() function of scikit-learn?
Harshalkumar's user avatar
2 votes
0 answers
230 views

Adding explicit user info to matrix factorization

In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this: $$ \hat{r}_{...
The Albatross's user avatar
1 vote
1 answer
203 views

Understanding the decomposed matrices in Singular Value Decomposition [duplicate]

I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
Victor M's user avatar
  • 309
1 vote
1 answer
472 views

Why is there a discrepancy between the eigenvalues of the covariance matrix (PCA) and the eigenvalues of the kernel matrix (kernel PCA)?

I've done PCA on my data matrix $ \mathbf{X} $ which gives me i.a. the eigenvalues $ \lambda $ and eigenvectors $ v $ of the data covariance matrix $ C=\mathbf{X}^T \mathbf{X} $. I'm now extending my ...
MaxGR's user avatar
  • 21
20 votes
1 answer
7k views

Relationship between Cholesky decomposition and matrix inversion?

I've been reviewing Gaussian Processes and, from what I can tell, there's some debate whether the "covariance matrix" (returned by the kernel), which needs to be inverted, should be done so ...
jbuddy_13's user avatar
  • 3,020
1 vote
0 answers
252 views

Alternating least squares --- what are the limitations?

I am taking an introductory course to Machine Learning and we learned alternating least squares for recommender systems. I learned that this method has some advantages --- easy to parallelize, and ...
Mar1129's user avatar
  • 11
2 votes
0 answers
46 views

Convex Optimization Algorithm for Additive Matrix Decomposition

Given an m $\times$ n matrix $Y = S + L + E$, where $S$ is a sparse matrix, $L$ is a low rank matrix and $E$ is a noise matrix, I want to recover $S$, $L$. One of the techniques given in Agarwal et al ...
Ismaac Newton's user avatar
1 vote
0 answers
244 views

Multivariate Normal Quadratic MGF: Eigendecomposition to Matrix form

If $X \sim \mathcal{N}(\mu, \Sigma)$ is a multivariate normal, then the quadratic $X^TAX$ has moment generating function $$M_{X^TAX}(t)= \frac{1}{\sqrt{\det(I - 2tA\Sigma)}}\exp\left(-\frac{1}{2}\mu'[...
ItsAllPurple's user avatar
1 vote
0 answers
91 views

Fastest way to find Leading singular value and vector (power iteration, rsvd etc)

I want to know the fastest way to find out the leading singular value and vector of a large rectangular matrix. I have seen 2 suggestions and have questions on both of them : Power Method : For this ...
Prateek Sasan's user avatar
2 votes
1 answer
263 views

How to explain the numerical discrepancy between FactoMineR::PCA() and the svd() in their output of the U matrix?

I am comparing the output of two functions in R to do Principal Component Analysis (PCA), the FactoMineR::PCA() and the ...
doctorate's user avatar
  • 1,105
3 votes
0 answers
402 views

Efficient computation of marginalized multivariate normal posterior distribution

In general,if we know that the marginal Gaussian distribution for some variable $\textbf{x}$ and a conditional Gaussian distribution for some $\textbf{y}|\textbf{x}$ of the forms: $$p(\textbf{x}) = \...
nwknoblauch's user avatar
0 votes
0 answers
520 views

Covariance Matrix Decomposition - Data Decorrelation

So I recently found out about Mahalanobis distance. Given a r.v $x$ in N-dimensional space, an associated metric is defined by $$M(x) = \sqrt{(x-\mu)^T S^{-1}(x-\mu)}$$ where $\mu$ and $S$ are mean ...
Aleksejs Fomins's user avatar
1 vote
1 answer
129 views

How to compute the left singular eigenvector matrix (U) from the output of prcomp() for PCA in R?

I am examining the output of the prcomp() function in R for PCA in light of the singular value decomposition equation: $X = U \cdot \Sigma \cdot V^{T}$, where: $X$: ...
doctorate's user avatar
  • 1,105

1
2 3 4 5 6