Just wondering if I am thinking of this correctly. I would really appreciate any comments on the approach.

I have a problem where I need to regress the outcome ($y$) on a design matrix ($X$). So,

$$ y | X \sim Normal(X\beta, \sigma) $$

However, I don't directly observe $X$ and rather I know that it comes from a distribution, so that:

$$ X \sim Normal(\hat{\mu}, \hat{\tau}) $$

Here, I have done some abuse of notation. $\hat{\mu}$ is a matrix and $\hat{\tau}$ is not really a covariance matrix, rather each column of $\hat{\tau}$ is a diagonal covariance matrix.

Perhaps a clearer way to writing would be:

$$ X[:,i] \sim Normal(\hat{\mu}[:,i],\hat{\tau}[:,i])$$

borrowing from MATLAB syntax.

My question is that does the following model make sense:

$$X \sim Normal(\hat{\mu}, \hat{\tau})$$

$$\beta \sim Dirichlet(\textbf{1})$$

$$y|X \sim Normal(X\beta, \sigma) $$

where $\hat{\mu}$ and $\hat{\tau}$ are known and we try to estimate $\beta$ (and $\sigma$).

If yes, what sort of model is this? Is this a generative model?

The reason this formulation confuses me is that since $X$ is not observed, does it make sense to regress onto a distribution of the design matrix?

Thank you for any suggestions!


It seems like you'd be interested in reading about Bayesian hierarchical modeling, as that seems well suited for your kind of problem where you know something about the hyperparameters and hyperpriors. I don't know too too much about it, but I'm sure there's loads of stuff online to better understand it, and your problem seems like a classic problem it deals with.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.