Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [matrix-decomposition]

Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.

0
votes
0answers
8 views

Eigendecomposition proof

Let $X$ be an $(n \times n)$ matrix. Let $V$ be the $(n \times n-k)$ be the matrix of eigenvectors of $X$ which correspond to non-zero eigenvalues of $X$. Let $E$ be the $(n-k \times n-k)$ diagonal ...
2
votes
0answers
20 views

Factorizing a matrix of distributions [on hold]

Let's say we have a matrix $X \in \mathbb{R}^{m \times n}$, then the (R-truncated) SVD allows to approximate: $X_{i,j} \approx \sum\limits_{r=1}^{R} \sigma_r \times U_{r,i} \times V_{r, j}$ Now I ...
0
votes
0answers
7 views

Implementation of Proximal alternating linearized minimization

The updates of the gradients are somehow wrong. I have implemented the below given algorithm. I have done something wrong ...
0
votes
0answers
12 views

Is the prediction with and without mean normalisation different in Collaborative Filtering?

In case of Collaborative Filtering: Given an output matrix I wish to learn parameters $\Theta$ (Parameter Vector) and X (Feature Vector). Now if I mean normalise the output matrix the values of $\...
1
vote
0answers
21 views

Why are some robust algorithms valid for Tucker decomposition, but not for CP decomposition?

I have been reading up about CP and Tucker decomposition. It makes sense that CP decomposition is a special case of Tucker decomposition, where the core tensor is super-diagonal. However, if this is ...
0
votes
0answers
11 views

AIC criteria for a matrix decomposition problem

I am trying to decompose a matrix such that $$A \approx UV_1 \approx UV_2V_1 \approx UV_3V_2V_1V_2$$ where $A \in R^{n \times l}$, $U \in R^{n \times k_1}$, $V_1 \in R^{k_1 \times l}$, $V_2 \in R^{k_2 ...
0
votes
0answers
31 views

Factorized matrix for recommendations, what then?

I have a dataset that looks like this: Image taken from this blog Let's assume that I have applied Matrix factorization and have learned the zero values for the items missing for every user. I now ...
0
votes
0answers
31 views

Rank 1 SVD with constraint on U

I need to perform a particular rank 1 decomposition of a sparse matrix $\mathbf{A} \in \mathbb{R}^{n\times n}$. In particular I am looking for the positive vector $\mathbf{u} \in \mathbb{R}^{+n}$ ...
0
votes
0answers
9 views

Improvements on using factorization machines?

I am fairly new to factorization machines, I have read papers about it and seen examples of it online. My current goal is to solve a recommendation problem and I'm not sure if what I'm doing is ...
3
votes
1answer
71 views

Which is more numerically stable for OLS: pinv vs QR

If I am doing standard OLS and want to calculate beta values (OLS estimators), which of the following is the more numerically stable method? And why? Assuming that the columns of $X$ are already mean-...
1
vote
0answers
26 views

Generalised least squares using QR decomposition

I know that the calculation of parameter values of a standard OLS can be made more efficient using a QR decomposition; i.e. if $X=QR$ and we are using the model $Y=X\beta+\epsilon$; Then it is true ...
3
votes
1answer
65 views

Orthogonal polynomials with respect to weighted inner product

Recently I have posted a question on SO, but maybe here is better place to ask. So, I have data and I want to fit polynomial of order $k$ orthogonal with respect to weighted inner product of functions:...
1
vote
0answers
44 views

What if the matrix is not invertible in the Alternating Least Squares Algorithm

We want to factorize $R_{n\times m}\approx X^{T}_{n\times k}Y_{k\times m}.$ The following update rules are given here: $$x_u = \left(\sum_{r_{ui}\in r_{u*}} y_i y_i^T + \lambda I_k\right)^{-1} \sum_{...
1
vote
0answers
34 views

How to make matrix factorization fast with sparse matrices? [closed]

Currently, I am working on a problem where I have to factorize a matrix $A\approx PQ$ which is sparse and has a dimension on the order of $1 \text{ million}^2.$ I have implemented the following ...
0
votes
0answers
24 views

Why Weighted Non Negative Matrix (WNMF) is better than standard NMF?

I have tried so hard to find answer of my subject query but have not found a single helpful source so far. Please let me know with the help of basic example that why WNMF is better than NMF in case of ...
1
vote
0answers
35 views

Using complex number in non-negative matrix factorization (NMF)

In short, I wonder which kind of data can use complex number for NMF. And could an imaginary part possibly be a vector? For detail, as I saw some papers used complex number in NMF (1), I think it ...
0
votes
0answers
33 views

Is there a problem with the “low-rank matrix approximation”?

I know that "rank" is the number or independent rows in the matrix and I know that the resulting matrix of the "low-rank matrix approximation" algorithm has lower rank than the original matrix. 1- But ...
0
votes
0answers
18 views

using matrix factorization for regression with many (only) categorical features

In recommendation systems one often uses matrix factorisation to estimate gaps in matrices, which represent the ratings of user of several movies (matrix user x movie). Let us say I have a regression ...
1
vote
1answer
41 views

Obtaining Dynamic Mode Decomposition modes

In this lecture prof. Kutz gives the Dynamic Mode Decomposition modes: $\Phi = X' V_r \Sigma_r^{-1} W $ which are the eigenvectors of the linear propagator matrix. This results from splitting the ...
-1
votes
1answer
71 views

Decomposition of a diagonal matrix

I want to decompose a diagonal matrix $\Lambda \in R^{n \times n}$ such that $$ \Lambda \approx A\Sigma A^T $$ where $\Sigma \in R^{k \times k}$ is a diagonal matrix and $A \in R^{n \times k}$ is a ...
0
votes
0answers
39 views

Normalizing sparse matrix by mean, should the mean be calculated excluding zero?

I have very sparse matrix (70% sparsity) which I want to normalize by mean. I tried using mean both include and exclude zero. The histogram between count (y-axis) and value (x-axis) shows The value ...
1
vote
0answers
57 views

Does non-Gaussian probabilistic PCA give orthogonal basis?

Probabilistic PCA - Gaussian: In their Probabilistic PCA model, Tipping and Bishop assume the following model $$ \boldsymbol{x} \sim \mathcal{N}(0, \mathbf{I})\\ \mathbf{t} | \boldsymbol{x} \sim \...
3
votes
1answer
646 views

Matrices: system that is “computationally singular” versus “exactly singular” [closed]

I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below: Error in ...
1
vote
1answer
34 views

Decomposition of vector into product of a function on a matrix and a function on a vector - Possible? [closed]

Say I have access to $N$-dim vector $Y$, $N \times p$ matrix $X$, and $q$-dim vector $Z$. Ultimately, I would like to learn the functions $g,f$ in: $\underset{N\times1}{\underbrace{Y}}=\underset{N\...
2
votes
1answer
89 views

Amount and sparsity of data for recommender systems

I'm starting to work in a project that will have a recommender system as one of its components. I'm trying to figure out if I have the right type of data for the recommender. The data contains ...
2
votes
0answers
383 views

Robust Anomaly Detection Algorithm from Netflix?

I have read a lot about the robust anomaly detection of Netflix which they open sourced as part of their Surus Project (https://github.com/Netflix/Surus). The project anomaly detector is based on the ...
7
votes
1answer
1k views

Difference between Cholesky decomposition and log-cholesky Decomposition

Is there any difference between a Cholesky decomposition and a log-cholesky decomposition? If yes, what is the difference? In the paper "An R package for dynamic linear models" by Giovanni Petris ( ...
1
vote
0answers
55 views

matrix factorization with non-negative constraint only on one of the factors

I have a 2D spectral data time series with a wavelength dimension and a time dimension, and I'd like to decompose it to the time evolution ($SV^T$ for SVD and $H$ for NNMF) of several spectral ...
0
votes
0answers
110 views

What is the relationship between latent factors in matrix factorization?

I am working on a project that involves using ALS to factor a m x n matrix $A$ into two latent matrices $UV$T, with dimensions m x k and n x k respectively. I was wondering, what is the relationship ...
3
votes
1answer
1k views

What is the relation between SVD and ALS?

I am trying to build a simple CF-recommender system using the small MovieLens data set. In order to do this, I tried to use ALS to factor my (user, item) matrix $A$ into a (user, latent-space) matrix $...
0
votes
0answers
17 views

Question regarding spectral decomposition of quadratic forms

Let $Y$~ $N(0,\Sigma)$ and A is non-negative definite symmetric matrix. Then $Q=Y^\prime AY$. I need to show that $Q=\sum_{i=1}^n \lambda_i Z_i^{2} $ ,where $Z_i^{2}=\chi_1^{2}$ (That means $Z_i$ ...
0
votes
0answers
43 views

Concurrent Time Series

I have a 3D spatial region of size X, Y & Z where each pixel (or voxel) in location $x$,$y$,$z$ has a time series of size $T\times 1$. Time series are highly (cross-)correlated with one another ...
3
votes
0answers
146 views

Nonnegative Matrix Factorization as Maximum Likelihood

Elements of Statistical Learning has this on such NMF loss function (section 14.6 Non-negative Matrix Factorization): The matrices $\mathbf{W}$ and $\mathbf{H}$ are found by maximizing $$ L(\mathbf{...
6
votes
2answers
1k views

Deriving Multiplicative Update Rules for NMF

How to derive the multiplicative update rules for the non-negative matrix factorization problem given by Lee and Seung. Minimize $\left \| V - WH \right \|^2$ with respect to $W$ and $H$, subject ...
0
votes
0answers
25 views

How to return the variance size of a recommendation system using SVD

One problem that many people have when making a recommendation system is the reasonableness of the suggestion / prediction, so I wanted to know how we can calculate the variance size of a generic ...
1
vote
0answers
398 views

Why is does the result square root of covariance matrix multiply with its transpose is the covariance matrix itself? [closed]

I think that if $P=\Sigma_u^\frac{1}{2}$, then it should be $PP=\Sigma_u$ instead of $\Sigma_u = PP'$. Then why is it $\Sigma_u = PP'$ (according to some literature)?
0
votes
1answer
310 views

Relationship between Alternating Least Squares and SVD

I have been assuming that ALS is simply an alternative algorithm for doing matrix decomposition that is more efficient, but in the end produces the same $U$ & $V$ matrices that SVD does. Is this ...
2
votes
0answers
49 views

Dealing with empty values in matrix decomposition

In matrix decomposition, given some matrix A we find 2 or 3 matrices which have less information than in A but when multiplied together as A' are optimally close to A in some sense. For instance the ...
1
vote
1answer
36 views

generating correlated samples

let's say I have n correlated variables, from which I would like to sample. I know there are several packages, like mvrnorm, ...
2
votes
1answer
108 views

ALS update rules in matrix form?

Question Is there a way to express the update rules of ALS (Alternating least squares): $$ u_i = \left(\sum_{j: (i, j) \in \Omega} v_j v_j^\top + \lambda I\right)^{-1} \sum_{j: (i, j) \in \Omega} M_{...
1
vote
1answer
719 views

is the difference of two positive definite matrices also positive definite?

If one subtracts one positive definite matrix from another, will the result still be positive definite, or not? How can one prove this?
2
votes
2answers
2k views
1
vote
0answers
282 views

SVD item similarity calculation

I am performing SVD on a rating matrix of Users and Items and I get 3 matrices out of which Vt provides latent feature for items. How do I compute similarities between a pair of items using these ...
0
votes
1answer
251 views

Is advanced linear algebra necessary for understanding multivariate statistics and stochastic processes?

I heard that linear algebra especially matrix algebra including singular value decomposition, symmetric, Hermitian, conjugate transpose, unitary geometry, transposes, and spectral theory show up in ...
1
vote
0answers
27 views

Theoretical form of the rank of the low rank approximation matrix

The low-rank matrix approximation problem involves finding of a rank k version of a m × n matrix A, labeled $A_k$, such that $A_k$ is as ”close” as possible to the best SVD approximation version of A ...
0
votes
0answers
66 views

What can be the reasons that L1-regularized NMF gets worse result than standard NMF in sparse matrix computation?

I apply L1-norm as a group sparsity constraint [1,2] into non-negative matrix factorization $V \approx WH$ for source separation. Objective functions: Standard NMF (Kullback-Leibler divergence): $...
5
votes
1answer
571 views

For symmetric matrices, is the Cholesky decomposition better than the SVD? [closed]

I am inverting a sparse, symmetric, ill-conditioned matrix. I have used both SVD and the LDL decomposition. I find that my results are better with the latter. Why? I understand that LDL ...
1
vote
1answer
60 views

Which recommender system algorithm should i use?

i need some advice for a project where i have to implement a recommender system for a market with very special characteristics. The scenario is as follows: Its a two-sided market, with buy and sell ...
2
votes
1answer
92 views

Matrix factorization for expanding matrix

In the paper Matrix Factorization Techniques for Recommender Systems Koren, Bell and Volinsky describe how the matrix $R_{n \times k}$ (users $\times$ movie ratings) can be decomposed to $P_{n \times ...
3
votes
0answers
300 views

What does the Cholesky decomposition of a correlation matrix tell you?

In this answer, the Cholesky decomposition of a correlation matrix is suggested as the means for testing for multicollinearity. I have a dataset that I am certain has high collinearity. I did the ...