3
$\begingroup$

Actually, this question is more or less a duplicate of the one which I have asked on math.stackexchange two days ago.

I did not get any answer there but I think now here is a better place to ask this question since it is more about "the philosophy", not the calculations involved in the concept, which is what the "math" board likes more.

I am trying to gain a good understanding of Bayesian Networks and the first thing I want to understand exactly is how they are built. I see that these networks are built on conditional independence assumptions in order to simplify joint distributions and they are built commonly by using causal relationships since they imply conditional independence given the direct causes. What I still don't understand is why these networks assume a Directed Acyclic (DAG) structure?

There can be systems which contain circular causality relationships. For example, let's think of a hypothetical machine consisting of two parallel plates which rub together. We think of three variables "Heat", "Plate Area Expansion" and "Friction". Plate Area Expansion is the effect of the cause "Heat" and "Friction" is just the effect of the plate area expansion since larger area means larger amount of friction, in turn. But if we think of it, "Friction" also causes an increase in the heat level, so "Friction" is a direct cause of "Heat" as well. This circular causality ends up with the following diagram:

Diagram

This is a Directed Cyclic Graph and violates acyclicity (DAG) assumptions on which the whole Bayesian Network idea is founded. So, how can be such systems with circular causes, feedbacks and loops represented with Bayesian Networks? It is surely not possible that the DAG theory does not support such systems because this "feedback" mechanism is a quite common and basic thing. But I just cannot see how those kinds of systems are represented with acyclic graphs and I am sure that I am missing something here. I want to know what I am just missing.

$\endgroup$
  • 3
    $\begingroup$ Replace the nodes of your graph with $Heat_t$, $PlateArea_t$, and $Friction_t$, where $t$ is the time step (0...N). The graph is now acyclic: $Heat_t$ points to $PlateArea_{t+1}$, $PlateArea_t$ points to $Friction_{t+1}$, and so on. The trick that makes this infinitely long network possibly tractable is the assumption that the parameters are the same across time, so all the lines connecting $Heat$ nodes to $PlateArea$ nodes have the same params (and so forth). Page 430 of this gives a good diagram for such a graph. $\endgroup$ – Stumpy Joe Pete Oct 19 '13 at 0:43
  • $\begingroup$ (1) Does this imply that causal systems always do evolve in time somehow? Because a Bayesian Network represents a joint probability distribution over the variables corresponding to its vertices, for example $P(Heat,PlateArea,Friction)$ if we do not take the time $t$ into account and this does not tell us anything about the chronological order. Let's assume that we start this hypothetical machine, wait for $t$ amounts of time and at time $t$ we instantaneously sample the system and get measurements for Heat, PlateArea and Friction. $\endgroup$ – Ufuk Can Bicici Oct 19 '13 at 20:07
  • $\begingroup$ (2)Lets assume again that we repeat this experiment (wait $t$ amount of time and sample instantaneously) many times more and we obtain a list of (Heat,PlateArea,Friction) measurements. Assuming all this experiments have been conducted independently, how we can set the causality relationships then? For just at an instant of time, where no chronological ordering exists, I cannot think of a causality relationship to build a Bayesian Network. So, what I am trying to understand is, do we need "time" to build Bayesian Networks, then? @StumpyJoePete $\endgroup$ – Ufuk Can Bicici Oct 19 '13 at 20:14
  • $\begingroup$ Thanks for your response, @StumpyJoePete. I'm not sure if the question is actually answered though. Rather than providing a solution to the issue of bidirectional causality, could you please provide an explanation of why it is not valid from a probabilistic point of view? $\endgroup$ – Vivek Subramanian Mar 5 '15 at 13:52
  • $\begingroup$ @VivekSubramanian Causality only goes forward in time (and yes, this implies that causal systems always do "evolve in time"). When we say "X causes Y", we mean that the value of X determines/influences the value of Y at some later time. And when we say "X and Y cause each other", we really mean there is feedback proceeding across time. I'm not saying there aren't formalisms that deal with undirected edges (there are), but in the formalism that the question uses, the edges must be directed and do imply a chronological ordering. $\endgroup$ – Stumpy Joe Pete Mar 5 '15 at 19:19
2
$\begingroup$

Just to add a little more clarity. this approach is sometimes called Temporal Bayesian Models. I have seen it being used in atleast one other situation of marketing mix models where today's marketing spend influences today's brand & revenue. Today's brand also influences tomorrow's brand & revenue and so on.

$\endgroup$
  • $\begingroup$ I have some questions about the usage of time for building a Bayesian Network, I added these as extra comments under my first question. @Learnerbeaver $\endgroup$ – Ufuk Can Bicici Oct 19 '13 at 20:21
  • $\begingroup$ Whether to use time or not should be dictated by your business hypothesis. Just to get some additional clarity, you could view the chapter on probabilistic graphical models by daphne koller in coursera.org. Might help. $\endgroup$ – Learnerbeaver Oct 20 '13 at 7:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.