Questions tagged [latent-variable]

Latent variables refer to variables that cannot be directly observed. These variable are defined in terms of observable variables. In narrow sense, "latent variable" is seen/modeled as what generates the observed variables in an implied data generation process. Also called hidden or lurking variables.

Filter by
Sorted by
Tagged with
41
votes
5answers
19k views

LDA vs word2vec

I am trying to understand what is similarity between Latent Dirichlet Allocation and word2vec for calculating word similarity. As I understand, LDA maps words to a vector of probabilities of latent ...
32
votes
3answers
39k views

Latent Class Analysis vs. Cluster Analysis - differences in inferences?

What are the differences in inferences that can be made from a latent class analysis (LCA) versus a cluster analysis? Is it correct that a LCA assumes an underlying latent variable that gives rise to ...
23
votes
5answers
17k views

How to get started with applying item response theory and what software to use?

Context I have been reading about item response theory, and I find it fascinating. I believe I understand the basics, but I am left wondering how to apply statistical techniques related to the area. ...
21
votes
1answer
2k views

Latent variable interpretation of generalized linear models (GLMs)

Short version: We know that logistic regression and probit regression can be interpreted as involving a continuous latent variable that gets discretized according to some fixed threshold prior to ...
16
votes
3answers
11k views

How to choose an optimal number of latent factors in non-negative matrix factorization?

Given a matrix $\mathbf V^{m \times n}$, Non-negative Matrix Factorization (NMF) finds two non-negative matrices $\mathbf W^{m \times k}$ and $\mathbf H^{k \times n}$ (i.e. with all elements $\ge 0$) ...
13
votes
1answer
6k views

Parameters vs latent variables

I have asked about this before and have really been struggling with identifying what makes a model parameter and what makes it a latent variable. So looking at various threads on this topic on this ...
12
votes
2answers
12k views

How to reduce number of items using factor analysis, internal consistency, and item response theory in conjunction?

I am in the process of empirically developing a questionnaire and I will be using arbitrary numbers in this example to illustrate. For context, I am developing a psychological questionnaire aimed at ...
10
votes
1answer
4k views

What is principal subspace in probabilistic PCA?

if $X$ is observed data matrix and $Y$ is latent variable then $$X=WY+\mu+\epsilon$$ Where $\mu$ is the mean of observed data, and $\epsilon$ is the Gaussian error/noise in data, and $W$ is called ...
10
votes
1answer
5k views

How do you use the EM algorithm to calculate MLEs for a latent variable formulation of a zero inflated Poisson model?

The zero inflated Poisson regression model is defined for a sample $(y_1,\ldots,y_n)$ by $$ Y_i = \begin{cases} 0 & \text{with probability} \ p_i+(1-p_i)e^{-\lambda_i}\\ k & \text{with ...
10
votes
2answers
4k views

How to compute the confidence intervals on regression coefficients in PLS?

The underlying model of PLS is that a given $n \times m$ matrix $X$ and $n$ vector $y$ are related by $$X = T P' + E,$$ $$y = T q' + f,$$ where $T$ is a latent $n \times k$ matrix, and $E, ...
10
votes
1answer
6k views

What's the difference between a MIMIC factor and a composite with indicators (SEM)?

In structural equation modeling with latent variables (SEM), a common model formulation is "Multiple Indicator, Multiple Cause" (MIMIC) where a latent variable is caused by some variables and ...
10
votes
1answer
426 views

What is the difference between VAE and Stochastic Backpropagation for Deep Generative Models?

What is the difference between Auto-encoding Variational Bayes and Stochastic Backpropagation for Deep Generative Models? Does inference in both methods lead to the same results? I'm not aware of any ...
9
votes
2answers
1k views

EM algorithm Practice Problem

This is a practice problem for a midterm exam. The problem is an EM algorithm example. I am having trouble with part (f). I list parts (a)-(e) for completion and in case I made a mistake earlier. Let ...
8
votes
4answers
1k views

Why aren't all tests scored via item analysis/response theory?

Is there a statistical reason why item analysis/response theory isn't more widely applied? For instance, if a teacher gives a 25 question multiple choice test and finds that 10 questions were answered ...
8
votes
2answers
635 views

Why is there a E in the name EM algorithm?

I understand where the E step happens in the algorithm (as explicated in the math section below). In my mind, the key ingenuity of the algorithm is the use of the Jensen's inequality to create a lower ...
8
votes
1answer
3k views

What are the primary differences between Taxometric analyses (e.g., MAXCOV, MAXEIG) and Latent Class analyses?

Recent research has attempted to determine if certain psychological constructs are latently dimensional or taxonic (i.e., including taxons or classes). For example, researchers may be interested in ...
8
votes
2answers
8k views

Which R package to use to conduct a latent class growth analysis (LCGA) / growth mixture model (GMM)?

I am trying to perform a latent class growth analysis (LCGA) and/or growth mixture models (GMMs) in R. The data I am using is an increasing number of forks of git repositories (discrete variable, not ...
8
votes
1answer
643 views

Why does probabilistic PCA use Gaussian prior over latent variables?

I am currently reading papers about probabilistic PCA and I am wondering why is Gaussian prior (and not some other prior) chosen for the latent variables? Is it just because it's simple or is there ...
7
votes
1answer
304 views

Are latent variable models modelling causality?

Is the purpose of latent variable models to model causality, where the causes are not observable i.e. latent? Are latent variables modelling causes of the observable variables?
7
votes
3answers
194 views

Estimating latent performance potential based on a sequence of observations

Context you have 200 observations of an individual's running time for the 100 metres measured once a day for 200 days. Assume the individual was not a runner before commencement of practice Based on ...
7
votes
1answer
386 views

Probabilistic models for partial least squares, reduced rank regression, and canonical correlation analysis?

This question results from the discussion following a previous question: What is the connection between partial least squares, reduced rank regression, and principal component regression? For ...
7
votes
1answer
2k views

Latent variables, overparameterization and MCMC convergence in bayesian models

Sometimes I have a large number of latent variables in a Bayesian hierarchical model to which, but I am only interested in estimating projected transformations of those latent variables (for example, ...
7
votes
2answers
3k views

When do you consider a variable is a latent variable?

The problem is to define when a variable might be considered as a latent variable. I am interested in how to describe a latent variable, and what are the properties of latent variables. My twofold ...
7
votes
2answers
3k views

Expectation maximization on Bayesian networks with latent variables

I am trying to determine parameters in a bayesian network with two latent variables (in blue). Every variable is discrete with 2-4 categories. The latent variables have 3 categories each. I am ...
6
votes
2answers
8k views

Student's t-test with a covariate?

I am testing the different between some variables $X$ and $Y$ using the Student's t-test. I suspect that there might be a latent variable $Z$ that has an effect on both $X$ and $Y$. How could I ...
6
votes
1answer
756 views

The fundamental theorem of simulation

The Fundamental Theorem of Simulation Simulating $X \sim f$ is equivalent to simulating $(X,U) \sim \mathscr{U}\{(x,u): 0<u<f(x)\}$ The proof is trivial. One thing that is made clear by ...
6
votes
1answer
3k views

Beginner references to understand probabilistic principal component analysis (PPCA)

I am totally new to machine learning. I started studying PCA from Jonathan Shlens, 2005, A Tutorial on Principal Component Analysis. The paper provides some concrete examples, and background ...
6
votes
2answers
337 views

Latent variables in Bayes nets with no physical interpretation

In Pattern Recognition and Machine Learning Bishop writes about Bayes networks: For practical applications of probabilistic models, it will typically be the highernumbered variables corresponding ...
6
votes
1answer
878 views

Semi-Hidden Markov Model with parameters of the emission probabilities depending on regressors

I'm trying to implement a Hidden Markov Model to detect past stockouts in sales (a stockout is when the retailer runs out of a product). It's probably better to say a Semi-Hidden Markov Model, let me ...
6
votes
1answer
344 views

How to calculate likelihood for a mixture model with missing data?

Toy explanation: I have set of different cars of different colours. There can be green, blue, red, etc. cars. I have a set of classes i.e.: "The set contains blue, red and pink cars" or "The set ...
6
votes
2answers
187 views

Sampling considerations in psychometric applications of item response theory

Background I have developed a recent interest in Item Response Theory and its applications. I am studying clinical psychology and am most interested in polytomous models aimed at modelling ...
5
votes
2answers
10k views

When a CFA model has a “covariance matrix was not positive definite” problem, is it due to the dataset or the model?

I am testing several CFA measurement models with Lavaan in R. The questionnaire that I am investigating has been shown to be composed of 1-factor, 3-factor, and 4-factor. In the dataset, I found ...
5
votes
2answers
1k views

Are there any good papers comparing different philosophical views of cluster analysis?

Lots of people use cluster analysis. I've heard very few explicitly say why. I imagine this is because within a given field, most researchers seem to understand why clustering is used for the problems ...
5
votes
2answers
108 views

What is the benefit of latent variables?

I have a model $p(x)$. How can adding latent variables $z$ help me? What are the main benefits of modelling $p(x, z)=p(x|z) p(z) $ instead of $p(x) $ alone? What would be some examples where modelling ...
5
votes
4answers
6k views

Why LDA (Latent Dirichlet Allocation) works (i.e. why put co-occurring words together)?

I am studying LDA, but have very weak statistical knowledge. I have a question regarding Gibbs sampling, one of the methods for inferring the distribution of topics and words-topic given a document, ...
5
votes
1answer
197 views

Toy regression question with latent variables

I originally asked this on a machine learning site, but one of the responses made me think that maybe this site is more suitable. Suppose you have two weighted coins, and every day you flip each one ...
5
votes
2answers
454 views

What is the motivation for the entropy term in the proof of EM algorithm?

Reading through the proof that EM algorithm monotonically increases the log-likelihood (until it converges), I noticed that the most important ingredient of the proof is the introduction of an entropy ...
5
votes
1answer
504 views

Is it possible to fit a multivariate regression model where the independent variable is latent?

I'm trying to fit a multivariate multiple regression model where the independent variable X is latent but I don't know where to start (I have prior information about the coefficient matrix so I can ...
5
votes
1answer
2k views

Selecting the number of mixtures / hidden states / latent variables

My question is regarding Gaussian Mixture models, Hidden Markov models (HMM) or any type of clustering or latent variable model, for which we can devise a likelihood function. Specifically, I train a ...
5
votes
1answer
2k views

What are the main differences between classical and Gibbs sampling Latent Dirichlet Allocations?

In these weeks I have been studying the classical Latent Dirichlet Allocation (LDA) algorithm by David Blei and colleagues (2003), and the LDA variant based on Gibbs sampling introduced by Tom ...
5
votes
1answer
317 views

IRT/Rasch modeling with very large N

I want to fit a 1-parameter IRT model on a questionaire with 15 questions and about six million people. Considering the large N, standard errors aren't essential. It looks like the IRT world is sort ...
5
votes
1answer
263 views

What is the difference in the latent space of a variational autoencoder and a regular autoencoder?

Should VAEs be even used for non-generative tasks? If I were to use both models for embedding images, how would the embedding space differ on a structural level?
5
votes
1answer
768 views

Given a topic distribution over words from LDA model how to calculate document distribution over topics for new document?

I'm using Spark 1.6.2 via the Python API. It seems that as of when this post is being written, the only data available from the LDA (latent Dirichlet allocation) model calculations is a topic ...
5
votes
0answers
447 views

Is it possible to get a prediction interval for logistic regression via a latent variable?

carbocation asked how to compute prediction intervals for logistic regression. The answer was that prediction intervals don't make sense for logistic regression because the response variable only ...
4
votes
2answers
753 views

Question about the latent variable in EM algorithm

In mixture models, Expectation maximization algorithm (EM) is a commonly used method to estimate the model parameters. Suppose that I have bivariate mixture model with two mixture components, with ...
4
votes
1answer
150 views

ELBO maximization with SGD

In cases such as Gaussian mixture models, there's is no closed-term solution for the original likelihood maximization. Maximizing the ELBO, however, does have analytical update formulas (i.e. formulas ...
4
votes
1answer
76 views

How to latently cluster regressors based on regressors' relationship with the outcome?

What is the best way/method to model patterns across coefficients and reduce number of coefficients based on these patterns? We have hundreds of regressors on the same scale and try to reduce the ...
4
votes
1answer
924 views

Change Score Model in lavaan

UPDATE: I think I was over-complicating my problem and am struggling through a new approach as described here: paired t-test as a simple latent change score model I am still accepting the answer ...
4
votes
2answers
2k views

Latent Dirichlet Allocation vs. pLSA

In the original LDA paper it is stated that: The parameters for a k-topic pLSI model are k multinomial distributions of size V and M mixtures over the k hidden topics. This gives kV +kM parameters ...
4
votes
1answer
257 views

Testing if a correlation between two variables depends on a continuous third variable

Consider a data frame with three variables: $x_1$, $x_2$, and $z_1$. I want to know if the correlation between the $X$ variables depends on $z_1$. Now, this could easily be done with an interaction ...

1
2 3 4 5
7