Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [matrix-decomposition]

Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.

0
votes
0answers
8 views

using matrix factorization for regression with many (only) categorical features

In recommendation systems one often uses matrix factorisation to estimate gaps in matrices, which represent the ratings of user of several movies (matrix user x movie). Let us say I have a regression ...
0
votes
0answers
19 views

Obtaining Dynamic Mode Decomposition modes

In this lecture prof. Kutz gives the Dynamic Mode Decomposition modes: $\Phi = X' V_r \Sigma_r^{-1} W $ This results from splitting the full data matrix $X$ into $X' = X(:,2:\text{end})$ and $X_1 = ...
0
votes
0answers
34 views

Decomposition of a diagonal matrix

I want to decompose a diagonal matrix $\Lambda \in R^{n \times n}$ such that $$ \Lambda \approx A\Sigma A^T $$ where $\Sigma \in R^{k \times k}$ is a diagonal matrix and $A \in R^{n \times k}$ is a ...
-1
votes
0answers
13 views

Variance proof for the stochastic estimator for trace of a matrix

I'm reviewing the proof the estimator of the trace of a matrix and am having trouble reconciling a jump in the proof of the variance of the estimator. The paper with the proof is found here. The ...
0
votes
0answers
11 views

Normalizing sparse matrix by mean, should the mean be calculated excluding zero?

I have very sparse matrix (70% sparsity) which I want to normalize by mean. I tried using mean both include and exclude zero. The histogram between count (y-axis) and value (x-axis) shows The value ...
1
vote
0answers
20 views

Does non-Gaussian probabilistic PCA give orthogonal basis?

Probabilistic PCA - Gaussian: In their Probabilistic PCA model, Tipping and Bishop assume the following model $$ \boldsymbol{x} \sim \mathcal{N}(0, \mathbf{I})\\ \mathbf{t} | \boldsymbol{x} \sim \...
2
votes
1answer
54 views

Matrices: system that is “computationally singular” versus “exactly singular” [closed]

I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below: Error in ...
1
vote
1answer
31 views

Decomposition of vector into product of a function on a matrix and a function on a vector - Possible? [closed]

Say I have access to $N$-dim vector $Y$, $N \times p$ matrix $X$, and $q$-dim vector $Z$. Ultimately, I would like to learn the functions $g,f$ in: $\underset{N\times1}{\underbrace{Y}}=\underset{N\...
2
votes
1answer
40 views

Amount and sparsity of data for recommender systems

I'm starting to work in a project that will have a recommender system as one of its components. I'm trying to figure out if I have the right type of data for the recommender. The data contains ...
0
votes
0answers
126 views

Robust Anomaly Detection Algorithm from Netflix?

I have read a lot about the robust anomaly detection of Netflix which they open sourced as part of their Surus Project (https://github.com/Netflix/Surus). The project anomaly detector is based on the ...
6
votes
1answer
682 views

Difference between Cholesky decomposition and log-cholesky Decomposition

Is there any difference between a Cholesky decomposition and a log-cholesky decomposition? If yes, what is the difference? In the paper "An R package for dynamic linear models" by Giovanni Petris ( ...
1
vote
0answers
35 views

matrix factorization with non-negative constraint only on one of the factors

I have a 2D spectral data time series with a wavelength dimension and a time dimension, and I'd like to decompose it to the time evolution ($SV^T$ for SVD and $H$ for NNMF) of several spectral ...
0
votes
0answers
31 views

What is the relationship between latent factors in matrix factorization?

I am working on a project that involves using ALS to factor a m x n matrix $A$ into two latent matrices $UV$T, with dimensions m x k and n x k respectively. I was wondering, what is the relationship ...
3
votes
1answer
350 views

What is the relation between SVD and ALS?

I am trying to build a simple CF-recommender system using the small MovieLens data set. In order to do this, I tried to use ALS to factor my (user, item) matrix $A$ into a (user, latent-space) matrix $...
0
votes
0answers
16 views

Question regarding spectral decomposition of quadratic forms

Let $Y$~ $N(0,\Sigma)$ and A is non-negative definite symmetric matrix. Then $Q=Y^\prime AY$. I need to show that $Q=\sum_{i=1}^n \lambda_i Z_i^{2} $ ,where $Z_i^{2}=\chi_1^{2}$ (That means $Z_i$ ...
0
votes
0answers
26 views

Concurrent Time Series

I have a 3D spatial region of size X, Y & Z where each pixel (or voxel) in location $x$,$y$,$z$ has a time series of size $T\times 1$. Time series are highly (cross-)correlated with one another ...
3
votes
0answers
95 views

Nonnegative Matrix Factorization as Maximum Likelihood

Elements of Statistical Learning has this on such NMF loss function (section 14.6 Non-negative Matrix Factorization): The matrices $\mathbf{W}$ and $\mathbf{H}$ are found by maximizing $$ L(\mathbf{...
6
votes
2answers
378 views

Deriving Multiplicative Update Rules for NMF

How to derive the multiplicative update rules for the non-negative matrix factorization problem given by Lee and Seung. Minimize $\left \| V - WH \right \|^2$ with respect to $W$ and $H$, subject ...
0
votes
0answers
13 views

How to return the variance size of a recommendation system using SVD

One problem that many people have when making a recommendation system is the reasonableness of the suggestion / prediction, so I wanted to know how we can calculate the variance size of a generic ...
1
vote
0answers
229 views

Why is does the result square root of covariance matrix multiply with its transpose is the covariance matrix itself? [closed]

I think that if $P=\Sigma_u^\frac{1}{2}$, then it should be $PP=\Sigma_u$ instead of $\Sigma_u = PP'$. Then why is it $\Sigma_u = PP'$ (according to some literature)?
0
votes
1answer
164 views

Relationship between Alternating Least Squares and SVD

I have been assuming that ALS is simply an alternative algorithm for doing matrix decomposition that is more efficient, but in the end produces the same $U$ & $V$ matrices that SVD does. Is this ...
2
votes
0answers
28 views

Dealing with empty values in matrix decomposition

In matrix decomposition, given some matrix A we find 2 or 3 matrices which have less information than in A but when multiplied together as A' are optimally close to A in some sense. For instance the ...
0
votes
1answer
29 views

generating correlated samples

let's say I have n correlated variables, from which I would like to sample. I know there are several packages, like mvrnorm, ...
2
votes
1answer
44 views

ALS update rules in matrix form?

Question Is there a way to express the update rules of ALS (Alternating least squares): $$ u_i = \left(\sum_{j: (i, j) \in \Omega} v_j v_j^\top + \lambda I\right)^{-1} \sum_{j: (i, j) \in \Omega} M_{...
0
votes
1answer
286 views

is the difference of two positive definite matrices also positive definite?

If one subtracts one positive definite matrix from another, will the result still be positive definite, or not? How can one prove this?
0
votes
0answers
21 views

Intuition of multi-colinearity and inflated $\beta$ — from a solver's perspective?

According to both my own experiments and the extensive literature on multi-colinearity, we know that the $\beta$ tends to get inflated when there is colinearity. Such inflation one of the main ...
0
votes
0answers
12 views

If algorithm is derived from majorization-minimization, is it correct to measure error from the majorized form?

The main objective function I wish to use for my algorithm is Kullback-Leibler divergence which can be majorized as: $D(V ||WH) \leq -\sum_{ij} V_{ij} \sum_{k} \pi_{ijk} log W_{ik} H_{kj} + \sum_{ij}(...
2
votes
2answers
827 views
1
vote
0answers
186 views

SVD item similarity calculation

I am performing SVD on a rating matrix of Users and Items and I get 3 matrices out of which Vt provides latent feature for items. How do I compute similarities between a pair of items using these ...
0
votes
1answer
176 views

Is advanced linear algebra necessary for understanding multivariate statistics and stochastic processes?

I heard that linear algebra especially matrix algebra including singular value decomposition, symmetric, Hermitian, conjugate transpose, unitary geometry, transposes, and spectral theory show up in ...
1
vote
0answers
23 views

Theoretical form of the rank of the low rank approximation matrix

The low-rank matrix approximation problem involves finding of a rank k version of a m × n matrix A, labeled $A_k$, such that $A_k$ is as ”close” as possible to the best SVD approximation version of A ...
0
votes
0answers
42 views

What can be the reasons that L1-regularized NMF gets worse result than standard NMF in sparse matrix computation?

I apply L1-norm as a group sparsity constraint [1,2] into non-negative matrix factorization $V \approx WH$ for source separation. Objective functions: Standard NMF (Kullback-Leibler divergence): $...
5
votes
1answer
430 views

For symmetric matrices, is the Cholesky decomposition better than the SVD? [closed]

I am inverting a sparse, symmetric, ill-conditioned matrix. I have used both SVD and the LDL decomposition. I find that my results are better with the latter. Why? I understand that LDL ...
0
votes
0answers
20 views

Improve non-negative matrix factorization with overly prior patterns for supervising basis matrix

My regularized NMF with L1 as a group sparsity constraint becomes worse than standard NMF because basis matrix has mismatched patterns in groups but I can't exclude them by theory. For example, I know ...
1
vote
1answer
54 views

Which recommender system algorithm should i use?

i need some advice for a project where i have to implement a recommender system for a market with very special characteristics. The scenario is as follows: Its a two-sided market, with buy and sell ...
2
votes
1answer
62 views

Matrix factorization for expanding matrix

In the paper Matrix Factorization Techniques for Recommender Systems Koren, Bell and Volinsky describe how the matrix $R_{n \times k}$ (users $\times$ movie ratings) can be decomposed to $P_{n \times ...
3
votes
0answers
248 views

What does the Cholesky decomposition of a correlation matrix tell you?

In this answer, the Cholesky decomposition of a correlation matrix is suggested as the means for testing for multicollinearity. I have a dataset that I am certain has high collinearity. I did the ...
0
votes
0answers
13 views

Join 2 NMF models where dictionaries has hierarchical structure

I have two NMF models $A = W_1A_{dict}$ and $B = W_2B_{dict}$ (where the $W$ represents weight coefficient matrix). What is a good way to join two NMFs if I know each column of $B_{dict}$ is summed up ...
1
vote
1answer
112 views

Dimension reduction methods: overview of complexity

For classical dimension reduction methods (PMF, PCA, SVD, t-SNE...) or some others, I need to know the complexity of efficient implementations: with $N$ vectors in dimension $d$ reduced to dimension ...
0
votes
0answers
119 views

How does “Automatic relevance determination” really work in context of matrix factorization?

I want to ask about the technique Automatic relevance determination for feature selection in this paper: https://hal.inria.fr/inria-00369376/document I do not under stand how large and small ...
0
votes
0answers
30 views

NMF: if activation pattern in $H$ is known, what constraint can be used in optimization $V=WH$?

I am quite new to non-negative matrix factorization (nmf) and would like to ask for the term of constraint I can use if I have pre-knowledge of activation pattern in matrix $H$ where $WH = V$ From ...
5
votes
1answer
562 views

Matrix factorization in recommender systems: adding a new user

I estimate ratings in a user-item matrix by decomposing the matrix into two matrices P and Q and then using gradient descent to ...
1
vote
0answers
227 views

how do i do the derivative of frobenius norm [duplicate]

I am trying to do matrix factorization. $ \frac{1}{2} \vert\vert X - WZ ^T \vert\vert ^2_F $ How do I find the derivative wrt W ? I am just told that it is $ W^T = (Z^TZ + \lambda I_K)^{-1}Z^TX^T $...
2
votes
0answers
45 views

Matrix Decomposition $ B = B^* + \sum_{i>1}\lambda_i B_i$

I recently encountered the following decomposition in a paper, may I know whether anyone on CV happened to know the name of the decomposition and a proof of it? $B$ is a regular stochastic matrix and ...
0
votes
0answers
19 views

In Singular value decomposition is there any way to relate singular values to the columns of the original matrix?

Given a Matrix $A$ where the SVD would be $$A= U \Sigma V^t$$ Where $\Sigma$ is a diagonal matrix with its singular values Assuming that I only had the $\Sigma$ values and I don't have the $U$ ...
5
votes
1answer
602 views

What's the relation between Matrix Factorization (MF) and Latent Dirichlet Allocation (LDA)?

My understanding is that both MF and LDA can be applied to do document classification. I will first summarize my understand about these two methods before I ask my questions. Assuming we use a big ...
1
vote
0answers
33 views

Approximate medoid using matrix approximation

The medoid of a set of $n$ points is defined as the point that minimizes the average distance to all the other points. If there is a matrix containing rows in which each row is the set of all metric ...
3
votes
1answer
143 views

Relationship between coupled matrix factorization and CCA

Canonical Correlation Analysis (CCA) computes a low-dimensional shared embedding of two set of variables $X$ and $Y$ such that the correlations among the variables between the two sets is maximized. ...
2
votes
0answers
40 views

Multi-Relational Learning via Tensor Factorization

Multi-relational data expressed in terms of a 3 order tensor, has been used to perform statistical relational learning (SRL) on heterogeneous networks using tensor factorization (eg. RESCAL). The ...
0
votes
0answers
2k views

What are eigenvalues and eigenvectors in factor analysis?

I understand the idea behind factor analysis, but everything I read on the topic seems to very vaguely cover the topic of eigenvalues and eigenvectors Whats the correct way to understand eigenvalues ...