How many parameters does the following Bayesian Network (Graphical model) contains?

Question

These are the relations defined in the graph.

P(D), P(E), P(A), P(C), P(G|D, E, A), P(I|E, A, C), P(T|E, C), P(F|E, A, C), P(S|A), P(J|G, I)

Hello, I have this network that shows a graph (Graphical model). The directed edges represent conditional probabilities. I need to compute a number of parameters for this whole graph. I know the formulas K - 1 where K is the number of states or K^M - 1 when I have M variables. The problem here is that the number of states each node can assume is different.

Here are the number of states each random variable can have.

[('Difficulty', 'D', 3), ('Effort', 'E', 3), ('Aptitute', 'A', 5), ('Confidence', 'C', 3), ('Grade', 'G', 5), ('Interview', 'I', 3), ('TuteAttendence', 'T', 3), ('ForumParticipation', 'F', 3), ('SAT', 'S', 3), ('Job', 'J', 2)]

How to compute a total number of parameters this model needs when fully connected vs when the DAG is considered.

I have made a python function that computes it as follows.

def calculate_params(n):
    n_states = len(n.states)
    if len(n.parents) == 0:
        return n_states
    else:
        p_states = []
        for p in n.parents:
            p_states.append(calculate_params(p) - 1)
        return (n_states - 1) * np.prod(np.asarray(p_states))

print(calculate_params(J))

it outputs 1923. Is this answer correct? Also how to compute the number of parameters when the graph is fully connected. In my opinion it should be print((3**7 - 1) + (5**2) + (2**1 - 1)) = 2212 but I am not sure.

What would be an easy correct way to compute this?

Answer 1

I think you are counting the variables incorrectly.

First, let us look at a simpler graph: $A \rightarrow B \rightarrow C$, and assume that $A$ has $5$, $B$ has $2$ and $C$ has $4$ possible states.

Let us first look at $A \rightarrow B$: For each value $A$ may take (which are 5), there are $2$ weights to specify (the probabilities for the values of $B$ given that $A$ takes a certain value). As the weights sum to one, this gives $2-1$ parameters each. Hence the $A \rightarrow B$ part is fully specified by $5 \cdot (2-1)$ many parameters. Similarly, the $B \rightarrow C$ part is specified by $2 \cdot (4-1)$ many parameters. Note that we do not have to calculate $5 (2-1) (4-1)$ or anything like that (which you are doing in your python code).

So in this simple example, the total number of parameters is $5\cdot(2-1) + 2\cdot(4-1) = 11$.

Now for your example. Let us first take the case of the fully connected graph. Then we simply have a probability measure on a space that has cardinality $3^7 \cdot 5^2 \cdot 2$. Thus we have that many weights to specify, but the weights have to sum to one, and hence there are $3^7 \cdot 5^2 \cdot 2 - 1 = 109349$ many parameters.

On the other hand, with your graph structure, we do the same as for the simple example above. For each node, we have to take the number of possible weights that need to be specified and multiply them by the number of points in the support of the parents. I.e., as a formula, we calculate:

$$ \sum_{n \in Nodes} (|\textrm{supp}(n)|-1) \cdot |\textrm{supp}(\textrm{parents}(n))| $$

Hereby, the support of all the parents is just the product of the respective supports of each parent node. And further, if a node has no parents, then we simply define $|\textrm{supp}(\textrm{parents}(n))|=1$.

Thus, plugging in the values for the given graph, we get (I use the order C, A, S, E, I, T, F, D, G, J) $$ (3-1)\\ + (5-1)\\ + (3-1)\cdot 2\\ + (3-1)\\ + (3-1)\cdot (3 \cdot 3)\\ + (3-1) \cdot (3\cdot3)\\ + (3-1)\cdot(3 \cdot 5 \cdot 3)\\ + (3-1)\\ + (5-1) \cdot (3 \cdot 3 \cdot 5)\\ + (2-1) \cdot 5 $$ which should (unless I made a mistake) be 325.