Questions tagged [matrix-decomposition]

Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Collaborative filtering movie recommender: how to account for missing ratings implying information about user preference?

I'm trying to learn about recommender systems with a fairly standard data set: I have a matrix with thousands of users, thousands of movies, and the ratings that users give to each movie. Obviously, ...
0
votes
0answers
12 views

How to diagnose convergence in CF for low-rank matrix factorization trained with SGD?

I have a huge dataset of user-item ratings and would like to make predictions. To this end I'm using the low-rank matrix factorization with SGD for CF (collaborative filtering). I pre-compute the ...
0
votes
0answers
22 views

Infinitesimal generator of Markov chain using numpy

I am computing the infinitesimal generator of a continuous-time Markov chain from the transition probabilities p. I am following the methodology described here, ...
1
vote
1answer
41 views

How can this L(2,1) problem be reduced to the orthogonal procrustes problem?

NOTE: Don't take this too serious -- the question is actually due to my misreading $\|y_i - Wx_i\|^2$ as $\|y_i - Wx_i\|_2$, see the answer. Smith et al. in Offline bilingual word vectors, orthogonal ...
0
votes
0answers
24 views

Relation between eigenvalues of original and transformed matrices

Let the matrix $X$ be some data arranged in rows. Consider the following eigenvalue decomposition $X^\top X = Q \Theta Q^\top=\sum_{i=1}^n \theta_iq_iq_i^\top$ where $q_i$ are the eigenvectors and $\...
2
votes
0answers
79 views

Square root of an almost diagonal matrix

Is there an efficient way to compute square root of an almost diagonal symmetric Hessian matrix, which is diagonal with the exception of the last two columns and last two rows? Could the efficient ...
1
vote
0answers
21 views

Enforcing constraints on weight matrices using ReLU activation

In the paper 'A Deep Non-Negative Matrix Factorization Neural Network' by Flunner and Hunter, proof of Theorem 1 says that "The ReLu Activation function is a standard approximation of a non-negative ...
0
votes
0answers
16 views

prove for Eckart-young in Frobenius norm

On page 74, linear algebra and learning from data. P74 the prove for Eckart-young in the frobenius norm. I couldnot understand why G = 0 in the proof, anybody can help me? Thank you!
0
votes
1answer
30 views

How to decompose covariance matrice, multiplied by constant, to sample from multivariate normal? [closed]

I need to sample from multivariate normal distribution with mean vector $\mu$ and covariance matrix $\Sigma_1$. For that I want to use the decomposition of $\Sigma_1$ into $UΛ{U}^T$ and samle as $\...
0
votes
0answers
19 views

Assessing the size of a cone by the singular values of $M$

Suppose I work with vectors from a high dimensional space with $100<N<1000$, e.g. word-embeddings. Say I have, already selected $R$ vectors, with $R\simeq10$, which form a matrix $M \in \mathbb{...
2
votes
0answers
38 views

Geometric interpretation of Cholesky Decomposition

I understand that a square matrix, say $A$, can be thought of as a linear transformation within the same space. I could be as simple as basis change or some other transformation. In this way of ...
0
votes
1answer
46 views

How to show that $X = LY$ where $Y\sim N(0,I)$?

Let $X\sim MVN(0,\Sigma)$ denote a random vector having the multivariate normal distribution with mean $0$ and covariance matrix $\Sigma$. Suppose we want to sample from $X\sim MVN(0,\Sigma)$. ...
1
vote
0answers
19 views

Difference between sparse cholesky and cholesky decomposition

I am confused as to the difference between the regular cholesky decomposition and the sparse cholesky decomposition (i.e. that which is performed by LAPACK's CHOLMOD...
0
votes
1answer
13 views

Is learning label embedding by factorizing label co-occurrence matrix unsupervised learning?

I was working on creating embeddings for medical concepts. These terms/phrases are used for annotating biomedical documents. Now usually the method of creating a co-occurrence matrix and then ...
0
votes
0answers
12 views

Is each row of latent factors obtained from matrix decomposition (SVD) dependent on the other rows of the higher dimensional matrix?

I implemented a recommendation system using user-user interaction data, learning missing ratings through alternating least squares and matrix factorization, which as I understand it, adjusts and ...
0
votes
0answers
38 views

Procedure to quickly find the near zero eigenvalues (and corresponding eigenvectors) of a positive semidefinite square matrix?

I have an ill-conditioned positive semidefinite $n \times n$ square matrix (Hessian), with unknown rank $r$, that I need to compute the inverse for. I'd prefer not to compute the SVD for performance ...
0
votes
0answers
31 views

Why does Alternating Least Squares (ALS) give us good results for missing values?

I was reading about the alternating least squares algorithm and could follow the math but somehow it didn't click for me. We start with random values for $U$ and $V$ and run the algorithm until we ...
0
votes
0answers
26 views

eigen value decomposition of co-variance a series generated by factor model

Let's assume $N\times T$ series $Y_t$ is generated by the following equation. $$ Y_t = \begin{bmatrix}A_x & A_m\end{bmatrix}\begin{bmatrix}x_t \\ m_t \end{bmatrix}$$ Where $A_x$ and $A_m$ are $N\...
0
votes
0answers
9 views

Why modeling known entries improves Tensor completion problem?

In the following paper: "Scalable tensor factorizations with missing data" by Evrim Acar Authors mention on page 2 that unlike conventional "CP" method which models both known and unknown values, they ...
1
vote
0answers
25 views

SVD versus RSVD

In the so-called incremental SVD used for collaborative filtering: http://www.machinelearning.org/proceedings/icml2007/papers/407.pdf http://www2.research.att.com/~volinsky/papers/ieeecomputer.pdf ...
0
votes
1answer
47 views

Interpretation of Eigenvalue vs. Singular Value plot

I'm doing some preliminary analysis on the feature matrix for a certain dataset (rows are observations, columns are feature dimensions). I have computed the SVD and PCA decompositions for this matrix ...
2
votes
0answers
42 views

Deriving Multiplicative Update Rules for Regularized NMF

After reading the following CrossValidated post, I cannot derived the correct multiplicative rules for regularized NMF from this paper. They obtain the coefficients $|I_u|$ and $|U_i|$ in the ...
1
vote
0answers
42 views

Symmetric decomposition of a covariance matrix that is not given explicitly

I am implementing a Monte-Carlo method that requires decomposition of a $k \times k$ covariance matrix $\Sigma=A^TA$ where the dimensionality of $A$ should also be $k\times k$. No further constraints ...
0
votes
0answers
15 views

QR decomposition used for estimating condition numbers

I have read that the QR decomposition is often used to estimate the condition number of a matrix but I don't understand... what is the benefit of using the QR decomposition for this? Is it purely ...
2
votes
0answers
22 views

Factorizing a matrix of distributions [closed]

Let's say we have a matrix $X \in \mathbb{R}^{m \times n}$, then the (R-truncated) SVD allows to approximate: $X_{i,j} \approx \sum\limits_{r=1}^{R} \sigma_r \times U_{r,i} \times V_{r, j}$ Now I ...
0
votes
1answer
21 views

Implementation of Proximal alternating linearized minimization

The updates of the gradients are somehow wrong. I have implemented the below given algorithm. I have done something wrong ...
0
votes
0answers
23 views

Is the prediction with and without mean normalisation different in Collaborative Filtering?

In case of Collaborative Filtering: Given an output matrix I wish to learn parameters $\Theta$ (Parameter Vector) and X (Feature Vector). Now if I mean normalise the output matrix the values of $\...
1
vote
0answers
30 views

Why are some robust algorithms valid for Tucker decomposition, but not for CP decomposition?

I have been reading up about CP and Tucker decomposition. It makes sense that CP decomposition is a special case of Tucker decomposition, where the core tensor is super-diagonal. However, if this is ...
0
votes
0answers
16 views

AIC criteria for a matrix decomposition problem

I am trying to decompose a matrix such that $$A \approx UV_1 \approx UV_2V_1 \approx UV_3V_2V_1V_2$$ where $A \in R^{n \times l}$, $U \in R^{n \times k_1}$, $V_1 \in R^{k_1 \times l}$, $V_2 \in R^{k_2 ...
0
votes
0answers
64 views

Factorized matrix for recommendations, what then?

I have a dataset that looks like this: Image taken from this blog Let's assume that I have applied Matrix factorization and have learned the zero values for the items missing for every user. I now ...
0
votes
0answers
37 views

Rank 1 SVD with constraint on U

I need to perform a particular rank 1 decomposition of a sparse matrix $\mathbf{A} \in \mathbb{R}^{n\times n}$. In particular I am looking for the positive vector $\mathbf{u} \in \mathbb{R}^{+n}$ ...
0
votes
0answers
23 views

Improvements on using factorization machines?

I am fairly new to factorization machines, I have read papers about it and seen examples of it online. My current goal is to solve a recommendation problem and I'm not sure if what I'm doing is ...
4
votes
1answer
382 views

Which is more numerically stable for OLS: pinv vs QR

If I am doing standard OLS and want to calculate beta values (OLS estimators), which of the following is the more numerically stable method? And why? Assuming that the columns of $X$ are already mean-...
1
vote
0answers
37 views

Generalised least squares using QR decomposition

I know that the calculation of parameter values of a standard OLS can be made more efficient using a QR decomposition; i.e. if $X=QR$ and we are using the model $Y=X\beta+\epsilon$; Then it is true ...
3
votes
1answer
92 views

Orthogonal polynomials with respect to weighted inner product

Recently I have posted a question on SO, but maybe here is better place to ask. So, I have data and I want to fit polynomial of order $k$ orthogonal with respect to weighted inner product of functions:...
1
vote
0answers
66 views

What if the matrix is not invertible in the Alternating Least Squares Algorithm

We want to factorize $R_{n\times m}\approx X^{T}_{n\times k}Y_{k\times m}.$ The following update rules are given here: $$x_u = \left(\sum_{r_{ui}\in r_{u*}} y_i y_i^T + \lambda I_k\right)^{-1} \sum_{...
1
vote
0answers
43 views

How to make matrix factorization fast with sparse matrices? [closed]

Currently, I am working on a problem where I have to factorize a matrix $A\approx PQ$ which is sparse and has a dimension on the order of $1 \text{ million}^2.$ I have implemented the following ...
0
votes
0answers
34 views

Why Weighted Non Negative Matrix (WNMF) is better than standard NMF?

I have tried so hard to find answer of my subject query but have not found a single helpful source so far. Please let me know with the help of basic example that why WNMF is better than NMF in case of ...
1
vote
0answers
58 views

Using complex number in non-negative matrix factorization (NMF)

In short, I wonder which kind of data can use complex number for NMF. And could an imaginary part possibly be a vector? For detail, as I saw some papers used complex number in NMF (1), I think it ...
0
votes
0answers
46 views

Is there a problem with the “low-rank matrix approximation”?

I know that "rank" is the number or independent rows in the matrix and I know that the resulting matrix of the "low-rank matrix approximation" algorithm has lower rank than the original matrix. 1- But ...
1
vote
1answer
66 views

Two ways of obtaining Dynamic Mode Decomposition modes - are they equivalent?

In this lecture prof. Kutz gives the Dynamic Mode Decomposition modes: $\Phi = X' V_r \Sigma_r^{-1} W $ which are the eigenvectors of the linear propagator matrix. This results from splitting the ...
-1
votes
1answer
87 views

Decomposition of a diagonal matrix

I want to decompose a diagonal matrix $\Lambda \in R^{n \times n}$ such that $$ \Lambda \approx A\Sigma A^T $$ where $\Sigma \in R^{k \times k}$ is a diagonal matrix and $A \in R^{n \times k}$ is a ...
1
vote
0answers
83 views

Normalizing sparse matrix by mean, should the mean be calculated excluding zero?

I have very sparse matrix (70% sparsity) which I want to normalize by mean. I tried using mean both include and exclude zero. The histogram between count (y-axis) and value (x-axis) shows The value ...
1
vote
0answers
80 views

Does non-Gaussian probabilistic PCA give orthogonal basis?

Probabilistic PCA - Gaussian: In their Probabilistic PCA model, Tipping and Bishop assume the following model $$ \boldsymbol{x} \sim \mathcal{N}(0, \mathbf{I})\\ \mathbf{t} | \boldsymbol{x} \sim \...
3
votes
1answer
1k views

Matrices: system that is “computationally singular” versus “exactly singular” [closed]

I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below: Error in ...
1
vote
1answer
40 views

Decomposition of vector into product of a function on a matrix and a function on a vector - Possible? [closed]

Say I have access to $N$-dim vector $Y$, $N \times p$ matrix $X$, and $q$-dim vector $Z$. Ultimately, I would like to learn the functions $g,f$ in: $\underset{N\times1}{\underbrace{Y}}=\underset{N\...
3
votes
1answer
203 views

Amount and sparsity of data for recommender systems

I'm starting to work in a project that will have a recommender system as one of its components. I'm trying to figure out if I have the right type of data for the recommender. The data contains ...
2
votes
0answers
652 views

Robust Anomaly Detection Algorithm from Netflix?

I have read a lot about the robust anomaly detection of Netflix which they open sourced as part of their Surus Project (https://github.com/Netflix/Surus). The project anomaly detector is based on the ...
6
votes
1answer
2k views

Difference between Cholesky decomposition and log-cholesky Decomposition

Is there any difference between a Cholesky decomposition and a log-cholesky decomposition? If yes, what is the difference? In the paper "An R package for dynamic linear models" by Giovanni Petris ( ...
2
votes
0answers
72 views

matrix factorization with non-negative constraint only on one of the factors

I have a 2D spectral data time series with a wavelength dimension and a time dimension, and I'd like to decompose it to the time evolution ($SV^T$ for SVD and $H$ for NNMF) of several spectral ...