Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [matrix-decomposition]

Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.

3
votes
1answer
58 views

Orthogonal polynomials with respect to weighted inner product

Recently I have posted a question on SO, but maybe here is better place to ask. So, I have data and I want to fit polynomial of order $k$ orthogonal with respect to weighted inner product of functions:...
1
vote
0answers
24 views

What if the matrix is not invertible in the Alternating Least Squares Algorithm

We want to factorize $R_{n\times m}\approx X^{T}_{n\times k}Y_{k\times m}.$ The following update rules are given here: $$x_u = \left(\sum_{r_{ui}\in r_{u*}} y_i y_i^T + \lambda I_k\right)^{-1} \sum_{...
1
vote
0answers
18 views

How to make matrix factorization fast with sparse matrices? [closed]

Currently, I am working on a problem where I have to factorize a matrix $A\approx PQ$ which is sparse and has a dimension on the order of $1 \text{ million}^2.$ I have implemented the following ...
0
votes
0answers
18 views

Why Weighted Non Negative Matrix (WNMF) is better than standard NMF?

I have tried so hard to find answer of my subject query but have not found a single helpful source so far. Please let me know with the help of basic example that why WNMF is better than NMF in case of ...
1
vote
0answers
17 views

Using complex number in non-negative matrix factorization (NMF)

In short, I wonder which kind of data can use complex number for NMF. And could an imaginary part possibly be a vector? For detail, as I saw some papers used complex number in NMF (1), I think it ...
0
votes
0answers
27 views

Is there a problem with the “low-rank matrix approximation”?

I know that "rank" is the number or independent rows in the matrix and I know that the resulting matrix of the "low-rank matrix approximation" algorithm has lower rank than the original matrix. 1- But ...
0
votes
0answers
15 views

using matrix factorization for regression with many (only) categorical features

In recommendation systems one often uses matrix factorisation to estimate gaps in matrices, which represent the ratings of user of several movies (matrix user x movie). Let us say I have a regression ...
1
vote
1answer
39 views

Obtaining Dynamic Mode Decomposition modes

In this lecture prof. Kutz gives the Dynamic Mode Decomposition modes: $\Phi = X' V_r \Sigma_r^{-1} W $ which are the eigenvectors of the linear propagator matrix. This results from splitting the ...
0
votes
0answers
38 views

Decomposition of a diagonal matrix

I want to decompose a diagonal matrix $\Lambda \in R^{n \times n}$ such that $$ \Lambda \approx A\Sigma A^T $$ where $\Sigma \in R^{k \times k}$ is a diagonal matrix and $A \in R^{n \times k}$ is a ...
0
votes
0answers
16 views

Normalizing sparse matrix by mean, should the mean be calculated excluding zero?

I have very sparse matrix (70% sparsity) which I want to normalize by mean. I tried using mean both include and exclude zero. The histogram between count (y-axis) and value (x-axis) shows The value ...
1
vote
0answers
34 views

Does non-Gaussian probabilistic PCA give orthogonal basis?

Probabilistic PCA - Gaussian: In their Probabilistic PCA model, Tipping and Bishop assume the following model $$ \boldsymbol{x} \sim \mathcal{N}(0, \mathbf{I})\\ \mathbf{t} | \boldsymbol{x} \sim \...
2
votes
1answer
137 views

Matrices: system that is “computationally singular” versus “exactly singular” [closed]

I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below: Error in ...
1
vote
1answer
31 views

Decomposition of vector into product of a function on a matrix and a function on a vector - Possible? [closed]

Say I have access to $N$-dim vector $Y$, $N \times p$ matrix $X$, and $q$-dim vector $Z$. Ultimately, I would like to learn the functions $g,f$ in: $\underset{N\times1}{\underbrace{Y}}=\underset{N\...
2
votes
1answer
55 views

Amount and sparsity of data for recommender systems

I'm starting to work in a project that will have a recommender system as one of its components. I'm trying to figure out if I have the right type of data for the recommender. The data contains ...
0
votes
0answers
240 views

Robust Anomaly Detection Algorithm from Netflix?

I have read a lot about the robust anomaly detection of Netflix which they open sourced as part of their Surus Project (https://github.com/Netflix/Surus). The project anomaly detector is based on the ...
6
votes
1answer
912 views

Difference between Cholesky decomposition and log-cholesky Decomposition

Is there any difference between a Cholesky decomposition and a log-cholesky decomposition? If yes, what is the difference? In the paper "An R package for dynamic linear models" by Giovanni Petris ( ...
1
vote
0answers
48 views

matrix factorization with non-negative constraint only on one of the factors

I have a 2D spectral data time series with a wavelength dimension and a time dimension, and I'd like to decompose it to the time evolution ($SV^T$ for SVD and $H$ for NNMF) of several spectral ...
0
votes
0answers
70 views

What is the relationship between latent factors in matrix factorization?

I am working on a project that involves using ALS to factor a m x n matrix $A$ into two latent matrices $UV$T, with dimensions m x k and n x k respectively. I was wondering, what is the relationship ...
3
votes
1answer
633 views

What is the relation between SVD and ALS?

I am trying to build a simple CF-recommender system using the small MovieLens data set. In order to do this, I tried to use ALS to factor my (user, item) matrix $A$ into a (user, latent-space) matrix $...
0
votes
0answers
17 views

Question regarding spectral decomposition of quadratic forms

Let $Y$~ $N(0,\Sigma)$ and A is non-negative definite symmetric matrix. Then $Q=Y^\prime AY$. I need to show that $Q=\sum_{i=1}^n \lambda_i Z_i^{2} $ ,where $Z_i^{2}=\chi_1^{2}$ (That means $Z_i$ ...
0
votes
0answers
35 views

Concurrent Time Series

I have a 3D spatial region of size X, Y & Z where each pixel (or voxel) in location $x$,$y$,$z$ has a time series of size $T\times 1$. Time series are highly (cross-)correlated with one another ...
3
votes
0answers
113 views

Nonnegative Matrix Factorization as Maximum Likelihood

Elements of Statistical Learning has this on such NMF loss function (section 14.6 Non-negative Matrix Factorization): The matrices $\mathbf{W}$ and $\mathbf{H}$ are found by maximizing $$ L(\mathbf{...
6
votes
2answers
647 views

Deriving Multiplicative Update Rules for NMF

How to derive the multiplicative update rules for the non-negative matrix factorization problem given by Lee and Seung. Minimize $\left \| V - WH \right \|^2$ with respect to $W$ and $H$, subject ...
0
votes
0answers
17 views

How to return the variance size of a recommendation system using SVD

One problem that many people have when making a recommendation system is the reasonableness of the suggestion / prediction, so I wanted to know how we can calculate the variance size of a generic ...
1
vote
0answers
321 views

Why is does the result square root of covariance matrix multiply with its transpose is the covariance matrix itself? [closed]

I think that if $P=\Sigma_u^\frac{1}{2}$, then it should be $PP=\Sigma_u$ instead of $\Sigma_u = PP'$. Then why is it $\Sigma_u = PP'$ (according to some literature)?
0
votes
1answer
232 views

Relationship between Alternating Least Squares and SVD

I have been assuming that ALS is simply an alternative algorithm for doing matrix decomposition that is more efficient, but in the end produces the same $U$ & $V$ matrices that SVD does. Is this ...
2
votes
0answers
36 views

Dealing with empty values in matrix decomposition

In matrix decomposition, given some matrix A we find 2 or 3 matrices which have less information than in A but when multiplied together as A' are optimally close to A in some sense. For instance the ...
1
vote
1answer
33 views

generating correlated samples

let's say I have n correlated variables, from which I would like to sample. I know there are several packages, like mvrnorm, ...
2
votes
1answer
65 views

ALS update rules in matrix form?

Question Is there a way to express the update rules of ALS (Alternating least squares): $$ u_i = \left(\sum_{j: (i, j) \in \Omega} v_j v_j^\top + \lambda I\right)^{-1} \sum_{j: (i, j) \in \Omega} M_{...
0
votes
1answer
473 views

is the difference of two positive definite matrices also positive definite?

If one subtracts one positive definite matrix from another, will the result still be positive definite, or not? How can one prove this?
0
votes
0answers
21 views

Intuition of multi-colinearity and inflated $\beta$ — from a solver's perspective?

According to both my own experiments and the extensive literature on multi-colinearity, we know that the $\beta$ tends to get inflated when there is colinearity. Such inflation one of the main ...
0
votes
0answers
13 views

If algorithm is derived from majorization-minimization, is it correct to measure error from the majorized form?

The main objective function I wish to use for my algorithm is Kullback-Leibler divergence which can be majorized as: $D(V ||WH) \leq -\sum_{ij} V_{ij} \sum_{k} \pi_{ijk} log W_{ik} H_{kj} + \sum_{ij}(...
2
votes
2answers
1k views
1
vote
0answers
210 views

SVD item similarity calculation

I am performing SVD on a rating matrix of Users and Items and I get 3 matrices out of which Vt provides latent feature for items. How do I compute similarities between a pair of items using these ...
0
votes
1answer
227 views

Is advanced linear algebra necessary for understanding multivariate statistics and stochastic processes?

I heard that linear algebra especially matrix algebra including singular value decomposition, symmetric, Hermitian, conjugate transpose, unitary geometry, transposes, and spectral theory show up in ...
1
vote
0answers
23 views

Theoretical form of the rank of the low rank approximation matrix

The low-rank matrix approximation problem involves finding of a rank k version of a m × n matrix A, labeled $A_k$, such that $A_k$ is as ”close” as possible to the best SVD approximation version of A ...
0
votes
0answers
61 views

What can be the reasons that L1-regularized NMF gets worse result than standard NMF in sparse matrix computation?

I apply L1-norm as a group sparsity constraint [1,2] into non-negative matrix factorization $V \approx WH$ for source separation. Objective functions: Standard NMF (Kullback-Leibler divergence): $...
5
votes
1answer
480 views

For symmetric matrices, is the Cholesky decomposition better than the SVD? [closed]

I am inverting a sparse, symmetric, ill-conditioned matrix. I have used both SVD and the LDL decomposition. I find that my results are better with the latter. Why? I understand that LDL ...
1
vote
1answer
58 views

Which recommender system algorithm should i use?

i need some advice for a project where i have to implement a recommender system for a market with very special characteristics. The scenario is as follows: Its a two-sided market, with buy and sell ...
2
votes
1answer
84 views

Matrix factorization for expanding matrix

In the paper Matrix Factorization Techniques for Recommender Systems Koren, Bell and Volinsky describe how the matrix $R_{n \times k}$ (users $\times$ movie ratings) can be decomposed to $P_{n \times ...
3
votes
0answers
264 views

What does the Cholesky decomposition of a correlation matrix tell you?

In this answer, the Cholesky decomposition of a correlation matrix is suggested as the means for testing for multicollinearity. I have a dataset that I am certain has high collinearity. I did the ...
1
vote
1answer
117 views

Dimension reduction methods: overview of complexity

For classical dimension reduction methods (PMF, PCA, SVD, t-SNE...) or some others, I need to know the complexity of efficient implementations: with $N$ vectors in dimension $d$ reduced to dimension ...
6
votes
1answer
739 views

Matrix factorization in recommender systems: adding a new user

I estimate ratings in a user-item matrix by decomposing the matrix into two matrices P and Q and then using gradient descent to ...
1
vote
0answers
238 views

how do i do the derivative of frobenius norm [duplicate]

I am trying to do matrix factorization. $ \frac{1}{2} \vert\vert X - WZ ^T \vert\vert ^2_F $ How do I find the derivative wrt W ? I am just told that it is $ W^T = (Z^TZ + \lambda I_K)^{-1}Z^TX^T $...
2
votes
0answers
46 views

Matrix Decomposition $ B = B^* + \sum_{i>1}\lambda_i B_i$

I recently encountered the following decomposition in a paper, may I know whether anyone on CV happened to know the name of the decomposition and a proof of it? $B$ is a regular stochastic matrix and ...
0
votes
0answers
20 views

In Singular value decomposition is there any way to relate singular values to the columns of the original matrix?

Given a Matrix $A$ where the SVD would be $$A= U \Sigma V^t$$ Where $\Sigma$ is a diagonal matrix with its singular values Assuming that I only had the $\Sigma$ values and I don't have the $U$ ...
5
votes
1answer
678 views

What's the relation between Matrix Factorization (MF) and Latent Dirichlet Allocation (LDA)?

My understanding is that both MF and LDA can be applied to do document classification. I will first summarize my understand about these two methods before I ask my questions. Assuming we use a big ...
1
vote
0answers
40 views

Approximate medoid using matrix approximation

The medoid of a set of $n$ points is defined as the point that minimizes the average distance to all the other points. If there is a matrix containing rows in which each row is the set of all metric ...
3
votes
1answer
158 views

Relationship between coupled matrix factorization and CCA

Canonical Correlation Analysis (CCA) computes a low-dimensional shared embedding of two set of variables $X$ and $Y$ such that the correlations among the variables between the two sets is maximized. ...
2
votes
0answers
43 views

Multi-Relational Learning via Tensor Factorization

Multi-relational data expressed in terms of a 3 order tensor, has been used to perform statistical relational learning (SRL) on heterogeneous networks using tensor factorization (eg. RESCAL). The ...