# Tag Info

## Hot answers tagged latent-variable

38 votes
Accepted

### Latent Class Analysis vs. Cluster Analysis - differences in inferences?

Latent Class Analysis is in fact an Finite Mixture Model (see here). The main difference between FMM and other clustering algorithms is that FMM's offer you a "model-based clustering" approach that ...
• 115k
29 votes
Accepted

### What is principal subspace in probabilistic PCA?

This is an excellent question. Probabilistic PCA (PPCA) is the following latent variable model \begin{align} \mathbf z &\sim \mathcal N(\mathbf 0, \mathbf I) \\ \mathbf x &\sim \mathcal N(\...
• 95.1k
20 votes
Accepted

### LDA vs word2vec

An answer to Topic models and word co-occurrence methods covers the difference (skip-gram word2vec is compression of pointwise mutual information (PMI)). So: neither method is a generalization of ...
• 5,616
15 votes
Accepted

### How to choose an optimal number of latent factors in non-negative matrix factorization?

To choose an optimal number of latent factors in non-negative matrix factorization, use cross-validation. As you wrote, the aim of NMF is to find low-dimensional $\mathbf W$ and $\mathbf H$ with all ...
• 95.1k
15 votes

### LDA vs word2vec

The two algorithms differ quite a bit in their purpose. LDA is aimed mostly at describing documents and document collections by assigning topic distributions to them, which in turn have word ...
• 2,542
14 votes
Accepted

### When a CFA model has a "covariance matrix was not positive definite" problem, is it due to the dataset or the model?

The covariance matrix of the data is always non-negative definite, there is no doubt about that. However, the model-implied covariance matrix may not be when some parameters take values outside their ...
• 29.4k
12 votes
Accepted

### Why is there a E in the name EM algorithm?

Expectations are central to the EM algorithm. To start with, the likelihood associated with the data $(x_1,\ldots,x_n)$ is represented as an expectation \begin{align*} p(x_1,\ldots,x_n;\theta) &= \...
• 92.7k
9 votes
Accepted

In the paper, and in general, (random) variables are everything which is drawn from a probability distribution. Latent (random) variables are the ones you don't directly observe ($y$ is observed, $\... • 2,696 9 votes Accepted ### What's the difference between a MIMIC factor and a composite with indicators (SEM)? They're the same model. It's useful to be able to define a latent variable as a composite outcome where that variable only has composite indicators. If you don't have: ... • 14.8k 9 votes ### Is it true that the word prior should be used only with latent random variables? The term prior (as well as posterior) is usually reserved for distributions defined in a Bayesian framework on objects that are not considered as random variables by other inferential approaches, ... • 92.7k 8 votes Accepted ### Beginner references to understand probabilistic principal component analysis (PPCA) PPCA was introduced in Tipping & Bishop, 1999, Probabilistic Principal Component Analysis. I would say that this paper itself is one of the best references: it is concise and clear. Nevertheless, ... • 95.1k 7 votes ### Student's t-test with a covariate? One common means of controlling for some other covariate would be via regression. Put the X and Y values into the response (DV), and a Y-group indicator (0 if in X, 1 if in Y) as a DV, along with your ... • 261k 7 votes Accepted ### The fundamental theorem of simulation Your question is unclear: when simulating$X\sim f(x)$, one can instead simulate$$(X,U)\sim\mathcal{U}(\{(x,u);\ 0<u<f(x)\})$$which has the joint density$$\mathbb{I}_{(0,f(x)}(u)\mathbb{I}_\... • 92.7k 6 votes ### How to choose an optimal number of latent factors in non-negative matrix factorization? To my knowledge, there are two good criteria: 1) the cophenetic correlation coefficient and 2) comparing the residual sum of squares against randomized data for a set of ranks (maybe there is a name ... 6 votes Accepted ### Probabilistic models for partial least squares, reduced rank regression, and canonical correlation analysis? Probabilistic canonical correlation analysis (probabilistic CCA, PCCA) was introduced in Bach & Jordan, 2005, A Probabilistic Interpretation of Canonical Correlation Analysis, several years after ... • 95.1k 6 votes Accepted ### ELBO maximization with SGD I think you confuse the purpose of the two methods. Maximizing the ELBO leads to a parameterized class of densities that approximates closely the true distribution, in terms of Kullback-Leibler ... • 1,196 6 votes ### Is it true that the word prior should be used only with latent random variables? Before answering your question, let's first explain some basic Bayesian mindset. In Bayesian statistics, everything is a random variable, the only difference between these random variables is whether ... • 663 6 votes Accepted ### Interpretating SEM estimates for 2 different datasets Small things first: I'm not sure what you mean by the high dimension of the dataset mattering. OLS is ML, so the fact that you used ML instead of OLS doesn't matter. The structural part of your model ... • 14.8k 5 votes Accepted ### Mean field variational inference The equation lists \prod_{i\not=j} q_i dz_i as not a constant because it's a multiplicative factor to q_i and is important from optimization perspective. Further down a formula (10.9) for q_i ... • 2,601 5 votes Accepted ### EM algorithm Practice Problem The complete data likelihood should not involve G! It should simply be the likelihood of \theta when the X's are exponential. Note that the complete data likelihood as you have it written ... • 2,871 5 votes Accepted ### Why does probabilistic PCA use Gaussian prior over latent variables? Probabilistic PCA Probabilistic PCA is a Gaussian latent variable model of the following form. Observations \mathbf x \in \mathbb R^D consist of D variables, latent variables \mathbf z \in \... • 95.1k 5 votes Accepted ### How and When to Use Marginalization in Stan Stan only samples from continuous parameter spaces, so for something like a finite mixture model, it is necessary to do marginalization to use Stan. On the other hand, if you have a hierarchical model ... • 1,396 5 votes Accepted ### Question about the latent variable in EM algorithm There is a lot of confusion in the question, confusion that could be reduced by looking at a textbook on the paper, or even the original 1977 paper by Dempster, Laird and Rubin. Here is an excerpt ... • 92.7k 5 votes Accepted ### Use Expectation-Maximization algorithm for obtaining maximal likelihood estimator You could express the complete log-likelihood as$$\ell(\theta) = -n\log\sigma_1-\frac{1}{2}\sum_{i=1}^n \sigma_1^{-2}(z_i-\mu_1)^2 -n\log\sigma^2-\frac{1}{2}\sum_{i=1}^n \sigma^{-2}(y_i-\beta_1-\... • 92.7k 5 votes ### Latent Profile Analysis and Statistical Methods in Psychology Latent Profile Analysis (LPA) is a term typically used for a model which identifies latent sub-populations within a population based on a certain set of categorical variables. In your case, you have ... • 49.8k 5 votes Accepted ### Finding a Common Thread in Disparate Indicators Have you considered the bifactor measurement model? From what I can glean from your question, it looks as though you are interested in a single social norms dimension, with an item bank that could ... • 420 4 votes Accepted ### Binary version of Probabilistic Matrix Factorization in pymc? You don't necessarily need to change$U$and$V$to be binary as well; in fact, doing so will probably be somewhat harmful. Remember that$\mathbb{E} R_{ij} = \sum_k U_{ik} V_{jk}$, so that if$U$and ... • 22.3k 4 votes ### Latent Class Analysis vs. Cluster Analysis - differences in inferences? A latent class model (or latent profile, or more generally, a finite mixture model) can be thought of as a probablistic model for clustering (or unsupervised classification). The goal is generally the ... • 3,683 4 votes ### Latent Class Analysis vs. Cluster Analysis - differences in inferences? The difference is Latent Class Analysis would use hidden data (which is usually patterns of association in the features) to determine probabilities for features in the class. Then inferences can be ... • 1,018 4 votes ### EM algorithm Practice Problem Based off @jsk's comments I will try to remedy my mistakes:$\begin{align*} L(\theta|X,G) &= \prod_{j=1}^n \theta e^{-\theta x_j} \end{align*}\begin{align*} Q(\theta,\theta^i) &= n\log{\...
• 8,672

Only top scored, non community-wiki answers of a minimum length are eligible