Questions tagged [matrix-decomposition]

Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.

Filter by
Sorted by
Tagged with
0 votes
0 answers
17 views

Decomposing Distance Matrix D for approximating Original Matrix A

Let's say we have a matrix $A \in R^{n \times d}$ where n is the number of elements and d is the dimension size. And we calculate the pairwise distances between each elements; say cosine for instance ...
user avatar
0 votes
0 answers
8 views

Find the principal component and the proportion of the total population variance explained by each when the variance covariance matrix is given

I can understand the part where we have to find the principal component from the variane covariance matrix- find eigen values, make eigen vector and normalise. The principal component would be ...
user avatar
1 vote
1 answer
56 views

Matrix dimensions in Linear Algebra vs Time series Analysis

I am confused or may misunderstand the dimensions of a Matrix when I was reading about time series analysis. From what I understand in linear Algebra, if we have a Matrix $A \in \mathbf{R}^{m*n}$, ...
user avatar
  • 13
0 votes
0 answers
88 views

Funk SVD for binary data - product like or dislike

Assume the following situation: you have a user-item sparse matrix. However, instead of the usual 1 to 5 rating scale, items can only receive a positive (1) or negative (-1) feedback. Thus, the matrix ...
user avatar
  • 133
0 votes
0 answers
7 views

Can a covariance matrix be decomposed into a single vector of meaningful distances?

I am interested in creating a single vector to describe a covariance matrix such that the most dissimilar variables are the most extreme values and the other variables have some numeric distance from ...
user avatar
0 votes
0 answers
6 views

CP Tensor Decomposition and Correlating Sample Magnitudes with Variables of Interest

I am learning about tensor decomposition, specifically CP, and am trying to understand if I can use it for my research. To give a bit more detail, I have brain imaging data from 10 participants, with ...
user avatar
  • 185
0 votes
0 answers
14 views

Proof of SVD generates equal eigenvectors as PCA [duplicate]

Figure link: https://people.cs.pitt.edu/~milos/courses/cs3750-Fall2014/lectures/class9.pdf In process of PCA, we either decompose covariance matrix, or do SVD on X. $$ C = \frac{1}{n-1} X^T X = \frac{...
user avatar
  • 1
0 votes
0 answers
34 views

What is the correlation between random variables after being multiplied by the same lower triangle matrix decomposed from a covariance matrix?

Assume $C_{n \times n}$ is a positive, symmetric and semi-definite covariance matrix, we know that the LU decomposition exists, i.e., $C_{n \times n}=L_{n \times n}U_{n \times n}$. Now $n$ ...
user avatar
0 votes
0 answers
27 views

Constrained Matrix Decomposition

I am working on a structural vector autoregression that requires imposing constraints on a matrix factorization. In particular, I have an N-dimensional positive definite matrix $\Sigma$ that I need to ...
user avatar
  • 2,127
0 votes
0 answers
32 views

Why does the ridge penalty shrink the singular values? [duplicate]

I am trying to understand the following analysis of ridge regression. I am new to SVD but I think I have a sufficient grasp on most of the content. There are two things I am struggling with. The ...
user avatar
  • 531
1 vote
1 answer
28 views

How to decompose a random walk (array) into its Markov Chain transition matrix?

The algorithm, PageRank, receives a Markov Chain transition matrix (page links from one to another.) Either by random walk, or more efficiently, eigenvectors, the stationary distribution of the Markov ...
user avatar
  • 1,650
1 vote
1 answer
35 views

Is the first independent component of independent component analysis always important?

I was looking at a neuroscience paper that used ICA to reduce dimensionality of calcium signaling profiles in 20 randomly selected neurons of a zebrafish brain. I presume that in Figure 2, ICA was ...
user avatar
  • 11
0 votes
0 answers
17 views

Why does Non-Negative Matrix Factorization reconstructs exactly the same matrix?

I'm trying recently to get into recommender systems and almost all tutorials I find mention collaborative filtering done with matrix factorization. I found this tutorial that describes how to build ...
user avatar
1 vote
0 answers
20 views

Relation between low-rank approximation, nuclear norm of a matrix and Singular Value Decomposition

I'm reading the following paper https://arxiv.org/pdf/2005.10203.pdf which proposes improvements on robustness of large graphs to defend against adversarial attacks that are nothing but slight ...
user avatar
0 votes
0 answers
51 views

How can I construct a desired variance-covariance matrix for simulating multivariate Gaussian distribution samples using MATLAB?

I want to simulate multivariate normal distribution samples to help understand PCA, biplot, etc. For example, I want to see how the correlation structure affects the appearance of 2-D biplot. Two ...
user avatar
0 votes
0 answers
17 views

Compressed sensing and matrix recovery

I'm reading a paper estimation of (near) low-rank matrices with noise and high-dimensional scaling by Negahban and Wainwright where authors defied compressed sensing as a particular instance of the ...
user avatar
  • 41
0 votes
0 answers
25 views

SVD on demeaned matrix

I'm trying to understand the effect of de-meaning with SVD. Suppose I have matrix $WW^T = \sum_{i=1}^n w_iw_i^T$ where $W$ is $n \times m$ and $w_i$ are its columns. Running SVD on this yields ...
user avatar
  • 141
1 vote
1 answer
17 views

Latent factors are the same in both decomposed matrices?

This question is in the context of recommendation systems. We can use matrix factorization techniques to decompose a user-product explicit/implicit matrix(R) into two matrices(U, P). Let's say R is a ...
user avatar
  • 643
0 votes
0 answers
197 views

Why absolute value of eigenvalues are used in PCA or LDA?

In PCA and LDA techniques, eigenvectors with the $k$ largest eigenvalues give principal components. However, when selecting these eigenvalues, are they to be sorted by the absolute value (regardless ...
user avatar
  • 1
0 votes
0 answers
18 views

can we use a hybrid optimization schem for NMF

The NMF problem of the form $$X \simeq WH$$ is a constrained biconvex optimization problem, and is often solved by alternating updates schemes. For example, the multiplicative update rules use ...
user avatar
0 votes
0 answers
21 views

How to estimate variance in Bayesian matrix factorization using Gibbs samples?

I have implemented a Gibbs sampler for Bayesian Matrix Factorization /Completion of matrix $R = (r_{ij})$ which is $(N, M)$ dimensional and $p(r_{ij} | \textbf{u}_i, \textbf{v}_j) = N(r_{ij}|\textbf{u}...
user avatar
  • 159
0 votes
0 answers
41 views

Drawing samples from matrix normal

I have to generate $n \times m$ sample ($A$) from a matrix normal distribution, given two covariance matrices: $n \times n$ row covariance matrix (matrix $B$) (defines the covariance between the rows ...
user avatar
  • 51
1 vote
0 answers
23 views

Are principal component analysis (PCA) and empirical orthogonal function (EOF) methods the same?

As far as I've seen, EOF is just PCA but instead of thinking about the data matrix X as (number of samples, number of features), you consider it as (number of time points, number of different spatial ...
user avatar
0 votes
1 answer
93 views

Matrix Factorization and Linear Regression

Which matrix factorization algorithm is used in LinearRegression() function of scikit-learn?
user avatar
1 vote
0 answers
120 views

Adding explicit user info to matrix factorization

In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this: $$ \hat{r}_{...
user avatar
1 vote
1 answer
59 views

Understanding the decomposed matrices in Singular Value Decomposition [duplicate]

I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
user avatar
  • 209
1 vote
1 answer
152 views

Why is there a discrepancy between the eigenvalues of the covariance matrix (PCA) and the eigenvalues of the kernel matrix (kernel PCA)?

I've done PCA on my data matrix $ \mathbf{X} $ which gives me i.a. the eigenvalues $ \lambda $ and eigenvectors $ v $ of the data covariance matrix $ C=\mathbf{X}^T \mathbf{X} $. I'm now extending my ...
user avatar
  • 21
17 votes
1 answer
3k views

Relationship between Cholesky decomposition and matrix inversion?

I've been reviewing Gaussian Processes and, from what I can tell, there's some debate whether the "covariance matrix" (returned by the kernel), which needs to be inverted, should be done so ...
user avatar
  • 1,650
1 vote
0 answers
136 views

Alternating least squares --- what are the limitations?

I am taking an introductory course to Machine Learning and we learned alternating least squares for recommender systems. I learned that this method has some advantages --- easy to parallelize, and ...
user avatar
  • 11
2 votes
0 answers
39 views

Convex Optimization Algorithm for Additive Matrix Decomposition

Given an m $\times$ n matrix $Y = S + L + E$, where $S$ is a sparse matrix, $L$ is a low rank matrix and $E$ is a noise matrix, I want to recover $S$, $L$. One of the techniques given in Agarwal et al ...
user avatar
1 vote
0 answers
132 views

Multivariate Normal Quadratic MGF: Eigendecomposition to Matrix form

If $X \sim \mathcal{N}(\mu, \Sigma)$ is a multivariate normal, then the quadratic $X^TAX$ has moment generating function $$M_{X^TAX}(t)= \frac{1}{\sqrt{\det(I - 2tA\Sigma)}}\exp\left(-\frac{1}{2}\mu'[...
user avatar
1 vote
0 answers
40 views

Fastest way to find Leading singular value and vector (power iteration, rsvd etc)

I want to know the fastest way to find out the leading singular value and vector of a large rectangular matrix. I have seen 2 suggestions and have questions on both of them : Power Method : For this ...
user avatar
1 vote
1 answer
118 views

How to explain the numerical discrepancy between FactoMineR::PCA() and the svd() in their output of the U matrix?

I am comparing the output of two functions in R to do Principal Component Analysis (PCA), the FactoMineR::PCA() and the ...
user avatar
  • 1,065
3 votes
0 answers
174 views

Efficient computation of marginalized multivariate normal posterior distribution

In general,if we know that the marginal Gaussian distribution for some variable $\textbf{x}$ and a conditional Gaussian distribution for some $\textbf{y}|\textbf{x}$ of the forms: $$p(\textbf{x}) = \...
user avatar
0 votes
0 answers
220 views

Covariance Matrix Decomposition - Data Decorrelation

So I recently found out about Mahalanobis distance. Given a r.v $x$ in N-dimensional space, an associated metric is defined by $$M(x) = \sqrt{(x-\mu)^T S^{-1}(x-\mu)}$$ where $\mu$ and $S$ are mean ...
user avatar
1 vote
1 answer
45 views

How to compute the left singular eigenvector matrix (U) from the output of prcomp() for PCA in R?

I am examining the output of the prcomp() function in R for PCA in light of the singular value decomposition equation: $X = U \cdot \Sigma \cdot V^{T}$, where: $X$: ...
user avatar
  • 1,065
1 vote
0 answers
24 views

How to find upper and lower bound

Let $\Sigma \in S_{++}^n$ be a symmteric positive definte matrix with all diagonal entries one. Let $U \in R^{n \times k_1}$, $W \in R^{n \times k_2}$, $\Lambda \in R^{k_1 \times k_1}$ and $T \in R^{...
user avatar
  • 185
1 vote
0 answers
63 views

Kernel matrix decomposition

I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post: Nystroem Method for Kernel Approximation Here, a ...
user avatar
  • 31
0 votes
0 answers
107 views

How to predict for test set when training a recommender by decomposing the utility matrix X=UV?

This probably sounds stupid but I don't get the workflow of building a recommending system by the utility matrix: X[i,j] = how much the ith user likes the jth object. For practical issues I refer to ...
user avatar
1 vote
0 answers
43 views

SVD for a complex data matrix — what is the meaning of the columns of $V$?

I've read this wonderful explanation of SVD, where the writer mentions that the columns of $V$ are the principal directions (Summary, #1). Is this also true when the data matrix $X$ is complex? If I'm ...
user avatar
  • 11
1 vote
1 answer
43 views

Convert the following expression w.r.t to the whole dataset instead of element of the dataset?

I am in the process of expressing the w in LSSVM with data points and constants. After I resolve the KKT conditions for the LSSVM I got $$w = \sum ^N _{i=1} \alpha_i x_i$$ Is it possible to convert ...
user avatar
4 votes
1 answer
807 views

Prove that sample covariance matrix is positive definite [duplicate]

Consider the $p \times p$ sample covariance matrix: $$\mathbf{S} = \frac{1}{n-1} \cdot \mathbf{Y}_\mathbf{c}^\text{T} \mathbf{Y}_\mathbf{c} \quad \quad \quad \mathbf{Y}_\mathbf{c} = \mathbf{C} \mathbf{...
user avatar
  • 211
3 votes
1 answer
439 views

Human intuition behind SVD in case of recommendation system

This does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the ...
user avatar
  • 625
10 votes
3 answers
955 views

SVD : Why right singular matrix is written as transpose

The SVD is always written as, A = U Σ V_Transpose The question is, Why is the right singular matrix written as V_Transpose? I mean lets say, W = V_Transpose and then write SVD as A = U Σ W SVD Image ...
0 votes
1 answer
196 views

Penalized Canonical Correlation in R with PMA Module

I am trying to use sparse canonical correlation analysis as implemented in the R PMA package. I'm finding that the correlations output by the package seem slightly inconsistent with the ones you would ...
user avatar
1 vote
0 answers
82 views

Symbolic Singular Value Decomposition? U,S,V as function of the elements of M [closed]

Suppose we want to compute the SVD of $\mathbf{M} = \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix}$ (...
user avatar
1 vote
1 answer
36 views

Interpreting PCA results of first two components

I don't like the looks of my PCA graph here. PCA coordinates should be uncorrelated, yet the variance between the coordinates of the second component increases as the first component decreases. What's ...
user avatar
  • 11
2 votes
0 answers
47 views

Use Matrix Factorization to predict probability of a recommendation system?

I have a dataset where I have a sparse utility matrix (user-product) with binary input: 1 if the user $i$ bought the product $j$, and 0 if it hasn't. However it has a different meaning on the test set,...
user avatar
  • 21
1 vote
0 answers
292 views

Is there a version of NMF that normalizes the sum of scores of each sample?

I want to decompose a nonnegative data matrix $A \in \mathbb{R}^{n\times m}$ into nonnegative basis vectors $U \in \mathbb{R}^{n \times k}$ and a score matrix $V \in \mathbb{R}^{m \times k}$ such that ...
user avatar
  • 111
2 votes
1 answer
87 views

ML model for Signal Decomposition [closed]

So recently I got a task which can be summarized as follows: Suppose we have 3 functions f1, f2, f3 and a certain combination of the functions gives us ...
user avatar

1
2 3 4 5 6