Disclaimer: the setup and specific example may not be a minimal example to illustrate the point, but I am not well-versed in these topics enough to construct a smaller example without accidentally omitting important details.

The setup

Consider a statistical model --Markov random field-- defined with respect to a graph $G=(V,E)$, where each node in the graph represents a random variable, and undirected edges represent dependencies. For example:

enter image description here

$A,B,D$ are dependent on each other, but $A|D$ is independent of $E$.

Now to define the model in question, supposed each variable, which we'll denote as $X_i$s for convenience, can take on values in $\mathcal{X}=\{1,2,3\}$.associate an edge a strictly positive $w_{s,t} \in \mathbb{R}_{>0}$ for $(s,t)\in E$. Given $G$, its joint mass function (assume discrete space for the random variables) is $$P(X) \propto \exp\{ \langle \boldsymbol \beta, \boldsymbol \phi(X) \rangle \}$$ Where sufficient statistics $\phi(X) = (\phi_{k})_{k \in \mathcal{X}\times\mathcal{X}}$, $\phi_k(X) = \sum_{e} w_e \boldsymbol 1_k(X_e)$, where $\boldsymbol{1}_k(X_e)$ is exchangeable in its argument. The parameters $\boldsymbol \beta$ are constrained to a simplex, i.e., $\sum_{k\in\mathcal{X}\times\mathcal{X}}\beta_k=1, \forall k\in\mathcal{X}\times\mathcal{X}, \beta_k > 0 $. Hopefully this setup is clear, please let me know if there is anything I should clarify.

A specific example

Consider two scenarios:

  1. I observe $10$ realizations $X^{(1)}, X^{(2)},\dots, X^{(10)}$ independently drawn from a model as defined above with respect to a graph $G = (V, E)$ where $|V| = 10$.
  2. I have one realization $X'^{(1)}$ drawn from a model as defined above with respect to a graph $G' = (V',E')$ where $|V'|=100$.

By writing out the sufficient statistics of the above two scenarios, we get a sum of the original sufficient statistics $\phi_{10}(X^{(1)}, X^{(2)},\dots, X^{(10)}) = \sum_{j=1}^{10} \phi(X^{(i)})$ for 1, and $\phi'(X')$ for 2.

My question

RE earlier disclaimer: I like to think the parallel to this setup is something along the lines of $n$ iid univariate Gaussian random variables versus one $n$-dimensional Gaussian random variable.

When I perform inference, posterior inference in my case, I would expect both models to have posteriors concentrate around their respective "truths". However, I am having trouble understanding how they differ as $|V'| = \sum_j |V^{(j)}| = n$ tends to infinity. On one hand, I think this is comparing apples to oranges in the sense that in one scenario (scenario 1), the interpretation of our parameter $\boldsymbol \beta$ pertains to a fixed graph, while scenario 2's parameter is interpretted as a parameter that dictates the dependencies of an increasing large graph...

Perhaps my question is a bit ill-formed, but I hope I communicated enough to express my concerns/confusion, and any discussion or comments will be really appreciated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.