I am trying to understand a neuroscience article by Karl Friston. In it he gives three equations that are, as I understand him, equivalent or inter-convertertable and refer to both physical and Shannon entropy. They appear as equation (5) in the article at http://www.fil.ion.ucl.ac.uk/spm/doc/papers/Action_and_behavior_A_free-energy_formulation.pdf (DOI 10.1007/s00422-010-0364-z). Here they are:

• Energy minus entropy: $F = −\langle\ln p(\stackrel{˜}{s},Ψ|m)\rangle_q + \langle\ln q(Ψ|μ)\rangle_q$

• Divergence plus surprise: $= D(q(Ψ|μ)||p(Ψ|\stackrel{˜}{s},m)) − \ln p (\stackrel{˜}{s}|m)$

• Complexity minus accuracy: $= D(q(Ψ|μ)||p(Ψ|m)) − \langle\ln p(\stackrel{˜}{s}|Ψ,m)\rangle_q$

The things I am struggling with at this point are 1) the meaning of the || in the 2nd and 3rd versions of the equations, 2) the negative logs. Any help in understanding how these equations are actually what Fristen claims them to be would be greatly appreciated. For example, in the 1st equation, in what sense is the first term energy, etc?

  • $\begingroup$ Are you sure this is the best stack exchange site for your question?? $\endgroup$
    – Steve S
    Jul 11, 2014 at 19:58
  • $\begingroup$ I'm not certain it is. I put it here because I gather that the equations are all about probabilities. I'll try a few others as well. $\endgroup$
    – johnhidley
    Jul 11, 2014 at 21:04
  • $\begingroup$ Cross-posted on Cognitive Sciences. $\endgroup$ Jul 11, 2014 at 23:48
  • $\begingroup$ Unless someone has an answer here, this might be better off on Mathematics. They know something about probabilities over there, too. $\endgroup$ Jul 12, 2014 at 6:59
  • $\begingroup$ Cross-posted on Mathematics $\endgroup$
    – honi
    Nov 16, 2015 at 19:52


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.