# Large? Number of parameters in MCMC model [closed]

I am implementing a Hierarchical Bayesian Modeling in order to model the relation between the independent and dependent parameters $(x, y)$.

I assume the relation is: $$y_i = \alpha + \beta x_i + \epsilon_i$$ where $\epsilon_i$ $\sim N(0, \sigma^2)$.

My joint probability distribution is:

$$p(\alpha, \beta, \sigma, \mu, \tau, x_t, y_t | D, I)$$ $$\propto p(\alpha, \beta, \sigma, \mu, \tau | I) p(x_t | \mu, \tau, I)p(y_t|\alpha, \beta, \sigma, x_t, I)$$ $$\times p(x|x_t, I)p(y|y_t, I)$$

where the observable data is $(x, y)$ and the true unobserved data is $(x_t, y_t)$.

$p(x_t | \mu, \tau, I)p(y_t|\alpha, \beta, \sigma, x_t, I) p(x|x_t, I)p(y|y_t, I) =$

$\prod\limits_{i = 1}^{N} \frac{1}{\sqrt{2\pi \tau^2}}\exp-\frac{(x_{ti} - \mu)^2}{2\tau^2} \times \frac{1}{\sqrt{2\pi \sigma^2}}\exp-\frac{(y_{ti} - \alpha - \beta x_{ti})^2}{2\sigma^2} \times \frac{1}{\sqrt{2\pi \sigma_{x,i}^2}}\exp-\frac{(x_i - x_{ti})^2}{2\sigma_{x,i}^2} \times \frac{1}{\sqrt{2\pi \sigma_{y,i}^2}}\exp-\frac{(y_i - y_{ti})^2}{2\sigma_{y,i}^2}$

# Calculate the number of parameters for the MCMC chain

number of parameters = 5 + len(yt) + len(xt) # 5: $\alpha, \beta, \sigma, \mu, \tau$


The number of items of $y_t$ = $x_t$ = number of data points

In my sample, I have 373 data points. So the number of parameters = 5 + 373 + 373 = 751!

Am I solving it correctly? This is my first attempt to do MCMC and to do HBM, is it normal to have HBM with 751 parameters?

# Edit

To make it clearer, I am trying to model the observed data points to the unobserved unknown data points. So for each data point $(x_i, y_i)$, there is an unknown true data point $(x_{ti}, y_{ti})$. So I am treating the data points $(x_t, y_t)$ as nuisance parameters. (This is why I added 373 + 373 to the number of parameters I have in my model).

## closed as unclear what you're asking by Tim♦, gung♦, usεr11852, John, Xi'anMay 10 '15 at 6:09

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

• It is not clear what do you mean by "unknown data". If there is no data, then there is no data, data cannot be unknown. – Tim May 9 '15 at 22:18

You do not assign every point a distribution, but you assign a distribution to your data.

So you want to find the parameters of the model that best captures your say 373 data points. Which according to your calculations is 5, not 5+373+373.

So you may want to re-edit your question.

Also what distribution do you use as a prior to your data?

If you do MCMC sampling and want to calculate the predictive posterior, cojugate distributions must be used so that it can be calculated in closed form.

You may want to exlore a bit more MCMC and HBM and get a better understanding of them before delving deeper. I know that citing wikipedia is not a good paradigm, but you can find a lot of stuff if you google bayesian methods.

Cheers, Panos.

• To make it clearer, if you take a closer look at my likelihood, I am trying to model the observed data points to the unobserved unknown data points. So for each data point $(x_i, y_i)$, there is an unknown true data point $(x_{ti}, y_{ti})$. So I am treating the data points $(x_t, y_t)$ as nuisance parameters. – aloha May 7 '15 at 22:13
• @po6 if there is a parameter for each data point and some extra parameters then such model is not estimable since you have more parameters then data. – Tim May 9 '15 at 22:14

It turned out that I was thinking correctly. The number of parameters in my model is equal to number of parameters I am interested to do an inference on plus the number of nuisance and hyperparameters.

In my case I am treating $\boldsymbol{\mathrm{x_t}}\ \textrm{and}\ \boldsymbol{\mathrm{y_t}}$ as nuisance parameters. (This is why I did 373 + 373)

(I am thinking from coding point of view). Ofcourse the number of parameters in my model (the parameters I am interested to inference) is equal to 3 !

• (-1) You are wrong. If your model is what you described, it has only $\alpha$, $\beta$ and $\epsilon$ parameters, plus hyperparameters for priors. There is no such a thing as parameters for "unknown data points", certainly not in simple regression model you are describing. It is not clear what do you mean by "true unobserved data", but it sounds that there is some misunderstanding of the model you are describing on your site. Even if you want to forecast or predict some data points using your model, you do not estimate parameters for individual points. – Tim May 9 '15 at 22:09