Expression for $p(\mathbf{s}|\mathbf{d})$ and the respective Markov Network

Question

I would like to check if I correctly derived my the expression for $p(\mathbf{s}|\mathbf{d})$. Here's the question:

Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in \{0,1\}$ is a binary random variable indicating whether the patient has $j$-th disease. A model for this is given by $$p(\mathbf{s,d})=\frac{1}{Z}\exp(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}+\mathbf{b}^T\mathbf{d})$$

where $Z$ is the normalisation constant and $\mathbf{W},\mathbf{a},\mathbf{b}$ are the parameters of the model.

I would like to draw the Markov Network for this model. Assume there are $n$ total symptoms and $m$ total diseases.

Question 1: Would this Markov Network be correct? Since it's a Markov Network, I don't have arrows, otherwise if it was a belief network, here's how I would draw it:

Question 2 Would this belief network be correct?

So now, I have the question:

Derive the expression for $p(\mathbf{s}|\mathbf{d})$. Is this distribution factorised?

Here's my solution:

$$p(\mathbf{s}|\mathbf{d}) = \frac{p(\mathbf{s},\mathbf{d})}{p(\mathbf{d})} = \frac{\exp\left(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}+\mathbf{b}^T\mathbf{d}\right)}{\sum_{\mathbf{s}}\exp\left(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}+\mathbf{b}^T\mathbf{d}\right)} = \frac{\exp\left(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}\right)}{\sum_{\mathbf{s}}\exp\left(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}\right)} \\ \propto \exp\left(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}\right) = \exp(\mathbf{s}^T\mathbf{W}d+\mathbf{s}^T\mathbf{a}) = \exp(\mathbf{s}^T(\mathbf{W}\mathbf{d}+\mathbf{a})) = \\ \exp\left(\sum_i^n\mathcal{1}_{\{s_i=1\}}\sum_j^m W_{ij}d_j+a_i\right) \\ \prod^n_i \exp(\sum_j^mW_{ij}d_j+a_i) = \prod_i^np(s_i|\mathbf{d}) $$

The distribution is factoried and $p(s_i|\mathbf{d}) = \sigma(a_i+\sum_jW_{ij}d_j)$ where $\sigma(x) = e^x/(1+e^x)$

Is my derivation correct?

Answer 1

$\newcommand{\sv}{\mathbf{s}} \newcommand{\dv}{\mathbf{d}} \newcommand{\Wv}{\mathbf{W}} \newcommand{\av}{\mathbf{a}} \newcommand{\bv}{\mathbf{b}} \newcommand{\pperp}{\perp\kern-5pt\perp} \newcommand{\npperp}{\not\perp\kern-8pt\perp} $ Your first question:
The probability $p(\mathbf s, \mathbf d)$ can be rewritten as follows: $$ \begin{align} p(\sv, \dv) &= \frac{1}{Z}\exp\left(\sv^T\Wv \dv + \av^T\sv + \bv^T\dv\right)\\ &= \frac{1}{Z}\prod_{ik}\exp(s_iW_{ik}d_k)\prod_r\exp(a_rs_r)\prod_{j}\exp(b_jd_j). \end{align} $$ Thus, you obtain exactly the factorization depicted by your Markov network (MN). There is an edge between $s_i$ and $d_k$ if and only if $W_{ik}\ne 0$.

Your second question:
We know now that your MN is describing conditional independence relation (CI) in a generic pdf of the given type, i.e. if the MN has a connection between two nodes, this would in the generic case yield dependence between the corresponding random variables. In particular, we deduce from the MN that in general: $$ \left(d_i \npperp d_k\right). $$ But in your belief network (BN), all the $s_r$ are colliders, thus the BN would require: $$ \left(d_i \pperp d_k\right), \quad i\ne k. $$ Therefore, your BN is not suitable in the sense that it is not an I-map for the generic pdf of the given family.

Your third question:
The fact that $p(\sv|\dv)$ factors in $\sv$ follows from the MN. Conditioning on $\dv$ completely separates this bipartite graph, so all the $d_i$ are independent, i.e. the pdf factors. Your derivation in formulas appears quite right, although I am a bit confused about the term $1_{s_i=1}$ and that the variable $\sv$ disappeared from the second to last expression. So I would suggest the following, slightly changed version of your derivation:
Following you, I will consider only factors containing $s_i$ and combine all factors containing $d_i$ into the overall constant factor: $$ \begin{align} p(\sv|\dv) &= \frac{p(\sv, \dv)}{p(\dv)}\\ &\propto \exp\left( \sv^T\Wv\dv + \av^T\sv\right)\\ &= \exp\left(\sv^T(\Wv\dv + \av) \right)\\ &= \prod_i \exp(s_i(\Wv\dv + \av)_i)\\ &= \prod_i f_i(s_i). \end{align} $$ This confirms that $p(\sv|\dv)$ does indeed factor.