Questions tagged [matrix-decomposition]

Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.

Filter by
Sorted by
Tagged with
0
votes
1answer
13 views

Does SVD provide the best low rank approximation for any matrix regardless of shape?

Wikipedia states (link below) that by the Eckart-Young-Mirsky theorem, the SVD provides the best low rank matrix approximation (on the basis of Frobenius norm of the error matrix) for any matrix A in ...
2
votes
1answer
38 views

How does multicollinearity affect the eigenvalues of a matrix?

I have been looking into ridge regression as a method to address multicollinearity in data. I am aware that multicollinearity can cause high variance in coefficient estimates. I have seen equations ...
5
votes
2answers
201 views

What is finite precision arithmetic and how does it affect SVD when computed by computers?

Was reading the paper "DETECTING AND ASSESSING THE PROBLEMS CAUSED BY MULTICOLLINEARITY:A USE OF THE SINGULAR-VALUE DECOMPOSITION" by David Belsley and Virginia Klema. After performing SVD, while ...
0
votes
2answers
26 views

How does the dimensions of my regression matrix effect its eigenvalules?

I was reading through another question answer where it discussed the inversion of the XX' matrix in ridge regression. It stated that it is impossible to have positive eigenvalues if matrix $XX'$(=$VDV'...
1
vote
0answers
7 views

Generalization Error and Matrix Factorizations?

This is more of a discussion/conceptual idea but: Is the notion of generalization error well-defined for a problem such as matrix factorization? I'm working with tensors/matrices and performing ...
0
votes
0answers
10 views

How do I approach this problem?

Let's say I have a dataset with multiple types of multiple ingredients (salt1,salt2, etc). Each n-th variation of each ingredient vs flavor may be represented by an n×k matrix that where an ingredient ...
3
votes
1answer
51 views

Moore Penrose Pseudo-Inverse Fast Algorithm in R [closed]

I want to apply Moore Penrose Pseudo-Inverse on my matrix, which is a 20,000 * 20,000 symmetric matrix with rank 19,999. I found ginv() function from the ...
1
vote
1answer
20 views

Why atoms in the dictionary of Dictionary Learning method are not required to be orthogonal?

According to Sparse Dictionary Learning (wiki), Sparse dictionary learning is a representation learning method which aims at finding a sparse representation of the input data (also known as sparse ...
0
votes
0answers
15 views

What should I be careful when using the word “supervised” in paper writing?

I am a biologist using machine learning tool for my research. I modified matrix decomposition ($V \approx WH$) to fit my data and wanted to describe about that in my paper. If I fixed one matrix ...
2
votes
0answers
12 views

How to approximate a Hermitian matrix with a transposed cross product of a single matrix?

I have a complex square matrix, and wish to learn latent factors (equally weighted latent factors, so not SVD) from this matrix. Given a Hermitian matrix A of ...
1
vote
1answer
88 views

Cholesky decomposition or alternative for negatively correlated data simulations

I want to generate some signals that have a correlation distribution around a specific pre-defined correlation value (i.e., the distribution of the values of their correlation matrix is around a ...
0
votes
0answers
31 views

Finding the optimal number of latent factors in Symmetric Singular Value Decomposition

Consider symmetric Singular Value Decomposition of a symmetric unitary matrix A into U, D, ...
0
votes
0answers
10 views

more then one user based Collaborative filtering

I want to get a movie recommendation by selecting multiple users. Usually, it takes one user ID and gives a recommendation. I thought user-based collaborative filtering with KNN is a good idea. I have ...
0
votes
0answers
9 views

How to get recommended users for an item with a matrix factorization recommender system?

Any tutorial or guide I've seen is for recommending items to a user, but how can I recommend users for an item? I am currently using the implicit library alternating least squares model. I have ...
0
votes
0answers
11 views

Intuitions behind Singular Value Decomposing generalizing eigen decomposition [duplicate]

Let M be mxn matrix then SVD of M will be UXW^* (sorry for X, assume summation). Then how does it generalizes eigen decomposition ? Since eigen decomposition is possible for nxn matrix and that are ...
0
votes
0answers
6 views

How to create linear user embedding from some answers to binary questions?

I have each user U_i answering 10 binary questions out of a pool Q with either answer 1 or 2. I would like to learn an embedding of user profiles based on these answers to predict is answer to other ...
5
votes
1answer
62 views

How to decompose covariance matrix for a bivariate normal distribution

Let $$\boldsymbol{x} = \begin{bmatrix} x\\ y\\ \end{bmatrix}, \boldsymbol{\mu} = \begin{bmatrix} \mu_x\\ \mu_y\\ \end{bmatrix}, \boldsymbol{\Sigma}=\begin{bmatrix} \sigma_x & \rho\sigma_x\sigma_y\\...
2
votes
0answers
91 views

Row similarity in matrix vs in different factorizations

Suppose an arbitrary $m \times n$ matrix $M$ and the factorizations: Arbitrary: $M = U_a V_a^T$, where $U_a$ is $m \times k$, $V_a$ is $n \times k$ ($k < m,n$), and $rank(U_a)=rank(V_a)=k$. SVD: $...
0
votes
0answers
9 views

Collaborative filtering movie recommender: how to account for missing ratings implying information about user preference?

I'm trying to learn about recommender systems with a fairly standard data set: I have a matrix with thousands of users, thousands of movies, and the ratings that users give to each movie. Obviously, ...
1
vote
0answers
52 views

Infinitesimal generator of Markov chain using numpy

I am computing the infinitesimal generator of a continuous-time Markov chain from the transition probabilities p. I am following the methodology described here, ...
1
vote
1answer
55 views

How can this L(2,1) problem be reduced to the orthogonal procrustes problem?

NOTE: Don't take this too serious -- the question is actually due to my misreading $\|y_i - Wx_i\|^2$ as $\|y_i - Wx_i\|_2$, see the answer. Smith et al. in Offline bilingual word vectors, orthogonal ...
0
votes
0answers
28 views

Relation between eigenvalues of original and transformed matrices

Let the matrix $X$ be some data arranged in rows. Consider the following eigenvalue decomposition $X^\top X = Q \Theta Q^\top=\sum_{i=1}^n \theta_iq_iq_i^\top$ where $q_i$ are the eigenvectors and $\...
2
votes
0answers
84 views

Square root of an almost diagonal matrix

Is there an efficient way to compute square root of an almost diagonal symmetric Hessian matrix, which is diagonal with the exception of the last two columns and last two rows? Could the efficient ...
1
vote
0answers
35 views

Enforcing constraints on weight matrices using ReLU activation

In the paper 'A Deep Non-Negative Matrix Factorization Neural Network' by Flunner and Hunter, proof of Theorem 1 says that "The ReLu Activation function is a standard approximation of a non-negative ...
0
votes
0answers
44 views

prove for Eckart-young in Frobenius norm

On page 74, linear algebra and learning from data. P74 the prove for Eckart-young in the frobenius norm. I couldnot understand why G = 0 in the proof, anybody can help me? Thank you!
0
votes
1answer
36 views

How to decompose covariance matrice, multiplied by constant, to sample from multivariate normal? [closed]

I need to sample from multivariate normal distribution with mean vector $\mu$ and covariance matrix $\Sigma_1$. For that I want to use the decomposition of $\Sigma_1$ into $UΛ{U}^T$ and samle as $\...
0
votes
0answers
19 views

Assessing the size of a cone by the singular values of $M$

Suppose I work with vectors from a high dimensional space with $100<N<1000$, e.g. word-embeddings. Say I have, already selected $R$ vectors, with $R\simeq10$, which form a matrix $M \in \mathbb{...
2
votes
0answers
79 views

Geometric interpretation of Cholesky Decomposition

I understand that a square matrix, say $A$, can be thought of as a linear transformation within the same space. I could be as simple as basis change or some other transformation. In this way of ...
0
votes
1answer
46 views

How to show that $X = LY$ where $Y\sim N(0,I)$?

Let $X\sim MVN(0,\Sigma)$ denote a random vector having the multivariate normal distribution with mean $0$ and covariance matrix $\Sigma$. Suppose we want to sample from $X\sim MVN(0,\Sigma)$. ...
1
vote
0answers
29 views

Difference between sparse cholesky and cholesky decomposition

I am confused as to the difference between the regular cholesky decomposition and the sparse cholesky decomposition (i.e. that which is performed by LAPACK's CHOLMOD...
0
votes
1answer
23 views

Is learning label embedding by factorizing label co-occurrence matrix unsupervised learning?

I was working on creating embeddings for medical concepts. These terms/phrases are used for annotating biomedical documents. Now usually the method of creating a co-occurrence matrix and then ...
0
votes
0answers
50 views

Is each row of latent factors obtained from matrix decomposition (SVD) dependent on the other rows of the higher dimensional matrix?

I implemented a recommendation system using user-user interaction data, learning missing ratings through alternating least squares and matrix factorization, which as I understand it, adjusts and ...
0
votes
0answers
42 views

Procedure to quickly find the near zero eigenvalues (and corresponding eigenvectors) of a positive semidefinite square matrix?

I have an ill-conditioned positive semidefinite $n \times n$ square matrix (Hessian), with unknown rank $r$, that I need to compute the inverse for. I'd prefer not to compute the SVD for performance ...
0
votes
0answers
69 views

Why does Alternating Least Squares (ALS) give us good results for missing values?

I was reading about the alternating least squares algorithm and could follow the math but somehow it didn't click for me. We start with random values for $U$ and $V$ and run the algorithm until we ...
0
votes
0answers
30 views

eigen value decomposition of co-variance a series generated by factor model

Let's assume $N\times T$ series $Y_t$ is generated by the following equation. $$ Y_t = \begin{bmatrix}A_x & A_m\end{bmatrix}\begin{bmatrix}x_t \\ m_t \end{bmatrix}$$ Where $A_x$ and $A_m$ are $N\...
0
votes
0answers
9 views

Why modeling known entries improves Tensor completion problem?

In the following paper: "Scalable tensor factorizations with missing data" by Evrim Acar Authors mention on page 2 that unlike conventional "CP" method which models both known and unknown values, they ...
1
vote
0answers
42 views

SVD versus RSVD

In the so-called incremental SVD used for collaborative filtering: http://www.machinelearning.org/proceedings/icml2007/papers/407.pdf http://www2.research.att.com/~volinsky/papers/ieeecomputer.pdf ...
0
votes
1answer
94 views

Interpretation of Eigenvalue vs. Singular Value plot

I'm doing some preliminary analysis on the feature matrix for a certain dataset (rows are observations, columns are feature dimensions). I have computed the SVD and PCA decompositions for this matrix ...
2
votes
0answers
60 views

Deriving Multiplicative Update Rules for Regularized NMF

After reading the following CrossValidated post, I cannot derived the correct multiplicative rules for regularized NMF from this paper. They obtain the coefficients $|I_u|$ and $|U_i|$ in the ...
1
vote
0answers
54 views

Symmetric decomposition of a covariance matrix that is not given explicitly

I am implementing a Monte-Carlo method that requires decomposition of a $k \times k$ covariance matrix $\Sigma=A^TA$ where the dimensionality of $A$ should also be $k\times k$. No further constraints ...
0
votes
0answers
16 views

QR decomposition used for estimating condition numbers

I have read that the QR decomposition is often used to estimate the condition number of a matrix but I don't understand... what is the benefit of using the QR decomposition for this? Is it purely ...
2
votes
0answers
23 views

Factorizing a matrix of distributions [closed]

Let's say we have a matrix $X \in \mathbb{R}^{m \times n}$, then the (R-truncated) SVD allows to approximate: $X_{i,j} \approx \sum\limits_{r=1}^{R} \sigma_r \times U_{r,i} \times V_{r, j}$ Now I ...
0
votes
1answer
33 views

Implementation of Proximal alternating linearized minimization

The updates of the gradients are somehow wrong. I have implemented the below given algorithm. I have done something wrong ...
1
vote
0answers
36 views

Why are some robust algorithms valid for Tucker decomposition, but not for CP decomposition?

I have been reading up about CP and Tucker decomposition. It makes sense that CP decomposition is a special case of Tucker decomposition, where the core tensor is super-diagonal. However, if this is ...
0
votes
0answers
106 views

Factorized matrix for recommendations, what then?

I have a dataset that looks like this: Image taken from this blog Let's assume that I have applied Matrix factorization and have learned the zero values for the items missing for every user. I now ...
0
votes
0answers
45 views

Rank 1 SVD with constraint on U

I need to perform a particular rank 1 decomposition of a sparse matrix $\mathbf{A} \in \mathbb{R}^{n\times n}$. In particular I am looking for the positive vector $\mathbf{u} \in \mathbb{R}^{+n}$ ...
4
votes
1answer
674 views

Which is more numerically stable for OLS: pinv vs QR

If I am doing standard OLS and want to calculate beta values (OLS estimators), which of the following is the more numerically stable method? And why? Assuming that the columns of $X$ are already mean-...
1
vote
0answers
46 views

Generalised least squares using QR decomposition

I know that the calculation of parameter values of a standard OLS can be made more efficient using a QR decomposition; i.e. if $X=QR$ and we are using the model $Y=X\beta+\epsilon$; Then it is true ...
3
votes
1answer
108 views

Orthogonal polynomials with respect to weighted inner product

Recently I have posted a question on SO, but maybe here is better place to ask. So, I have data and I want to fit polynomial of order $k$ orthogonal with respect to weighted inner product of functions:...
1
vote
0answers
73 views

What if the matrix is not invertible in the Alternating Least Squares Algorithm

We want to factorize $R_{n\times m}\approx X^{T}_{n\times k}Y_{k\times m}.$ The following update rules are given here: $$x_u = \left(\sum_{r_{ui}\in r_{u*}} y_i y_i^T + \lambda I_k\right)^{-1} \sum_{...

1
2 3 4 5