0
$\begingroup$

Disclaimer: the setup and specific example may not be a minimal example to illustrate the point, but I am not well-versed in these topics enough to construct a smaller example without accidentally omitting important details.

The setup

Consider a statistical model --Markov random field-- defined with respect to a graph $G=(V,E)$, where each node in the graph represents a random variable, and undirected edges represent dependencies. For example:

enter image description here

$A,B,D$ are dependent on each other, but $A|D$ is independent of $E$.

Now to define the model in question, supposed each variable, which we'll denote as $X_i$s for convenience, can take on values in $\mathcal{X}=\{1,2,3\}$.associate an edge a strictly positive $w_{s,t} \in \mathbb{R}_{>0}$ for $(s,t)\in E$. Given $G$, its joint mass function (assume discrete space for the random variables) is $$P(X) \propto \exp\{ \langle \boldsymbol \beta, \boldsymbol \phi(X) \rangle \}$$ Where sufficient statistics $\phi(X) = (\phi_{k})_{k \in \mathcal{X}\times\mathcal{X}}$, $\phi_k(X) = \sum_{e} w_e \boldsymbol 1_k(X_e)$, where $\boldsymbol{1}_k(X_e)$ is exchangeable in its argument. The parameters $\boldsymbol \beta$ are constrained to a simplex, i.e., $\sum_{k\in\mathcal{X}\times\mathcal{X}}\beta_k=1, \forall k\in\mathcal{X}\times\mathcal{X}, \beta_k > 0 $. Hopefully this setup is clear, please let me know if there is anything I should clarify.

A specific example

Consider two scenarios:

  1. I observe $10$ realizations $X^{(1)}, X^{(2)},\dots, X^{(10)}$ independently drawn from a model as defined above with respect to a graph $G = (V, E)$ where $|V| = 10$.
  2. I have one realization $X'^{(1)}$ drawn from a model as defined above with respect to a graph $G' = (V',E')$ where $|V'|=100$.

By writing out the sufficient statistics of the above two scenarios, we get a sum of the original sufficient statistics $\phi_{10}(X^{(1)}, X^{(2)},\dots, X^{(10)}) = \sum_{j=1}^{10} \phi(X^{(i)})$ for 1, and $\phi'(X')$ for 2.

My question

RE earlier disclaimer: I like to think the parallel to this setup is something along the lines of $n$ iid univariate Gaussian random variables versus one $n$-dimensional Gaussian random variable.

When I perform inference, posterior inference in my case, I would expect both models to have posteriors concentrate around their respective "truths". However, I am having trouble understanding how they differ as $|V'| = \sum_j |V^{(j)}| = n$ tends to infinity. On one hand, I think this is comparing apples to oranges in the sense that in one scenario (scenario 1), the interpretation of our parameter $\boldsymbol \beta$ pertains to a fixed graph, while scenario 2's parameter is interpretted as a parameter that dictates the dependencies of an increasing large graph...

Perhaps my question is a bit ill-formed, but I hope I communicated enough to express my concerns/confusion, and any discussion or comments will be really appreciated.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.