4
$\begingroup$

Background: As I understand the role of probability in Quantum Mechanics, the idea is that no observable event can have negative probability, but that it can make sense for unobserved quantities to have negative probability, so long as when marginalizing over unobserved events we obtain a proper probability distribution over observable ones.

Bayesian hierarchical models often contain unobservable latent variables. Conceivably, endowing some of these latent variables negative probability (densities) could still result in a well defined probability density over the observable nodes. And, also conceivably, doing so would allow for a more parsimonious description of the marginal probability distribution than is possible using latent variables with positive probabilities only.

More Background: Maybe an example is in order. Let's consider the standard two-cluster Gaussian mixture model with unknown mean and identical, known variance in dimension 1, for now with positive probabilities as usual:

$$z \sim bern(\rho)$$

$$y|z=0 \sim N(\mu_0,1)$$

$$y|z=1 \sim N(\mu_1,1)$$

Here, the unobserved latent variable is $z$, and when marginilizing over it, we are left with:

$$\delta_y(y) = \rho \delta_{N(\mu_0,1)}(y) + (1-\rho) \delta_{N(\mu_1,1)}(y)$$

where $\delta_y$ is the density of $y$ and $\delta_{N(a,b)}$ gives the density of a normal distribution with mean $a$ and variance $b$.

Here's what that density looks like for $\mu_0=1$,$\mu_1=-1$ and $\rho=0.9$:

standard mixture

(Since the peaks are close relative to the standard error it looks much like a single normal distribution).

Now I'm going to change the second density from being a $N(\mu_1,1)$ to a distribution with negative probabilities by changing it's density to this function:

$$\delta_{\aleph} := \delta_{N(\mu_1,0)}\textrm{sgn}(\mu_1-y)$$

where $\textrm{sgn}(a)$ is the sign function, so it looks like this: negative density

If we just plug this into the previous expression for the marginal density, we get:

$$\delta_{-}(y) = \rho \delta_{N(\mu_0,1)}(y) + (1-\rho) \delta_{\aleph}(y)$$

which looks like this

negative marginal

This is by all accounts a proper marginal: if I had thought of it, I could happily use it as a likelihood for my data without any need for negative probabilities by directly applying this distribution.

Question: Are there examples of negative densities being used in the context of latent variable models as a modeling device in order to define a generative model of observed data?

Such a thing exists in the context of the outcome of quantum phenomena, such as the double slit experiment. See this Wikipedia article for more on negative probabilities.

$\endgroup$
20
  • 5
    $\begingroup$ There is no such thing as a negative probability. Such instances of the term are a misuse of terminology. $\endgroup$
    – Galen
    Commented Nov 12, 2021 at 17:07
  • 4
    $\begingroup$ Allow me to direct you to review Galen's comment. There is no such thing as a negative probability. The concept as you've defined it does not exist. Probabilities are defined as non-negative: en.wikipedia.org/wiki/Probability_measure . You can achieve the same effect using valid probability distributions. $\endgroup$
    – jbowman
    Commented Nov 12, 2021 at 17:14
  • 3
    $\begingroup$ I have used linear combinations of probability distributions with negative coefficients in several posts here on CV. For instance, my expression for the distribution of a sum of Gamma variables can have negative coefficients. I have also provided explanations of phenomena that involve adding negative density values to densities, such as stats.stackexchange.com/a/299765/919. Are these the sorts of things you are looking for? $\endgroup$
    – whuber
    Commented Nov 12, 2021 at 17:41
  • 3
    $\begingroup$ @Galen Also, I was inclined to take you at your word when you said"there is no such thing as a negative probability". But I'm struggling now that I see there is a wikipedia page with title "Negative Probability". And a recent article dealing with it: papers.ssrn.com/sol3/papers.cfm?abstract_id=1773077 And that Paul Dirac said (according to the Wiki) "Negative energies and probabilities should not be considered as nonsense. They are well-defined concepts mathematically, like a negative of money." Is there a movement I haven't run into that seeks to reject all this? $\endgroup$ Commented Nov 12, 2021 at 18:09
  • 3
    $\begingroup$ Thank you--that's a helpful reference and remark. As far as I can tell, that portion of the Wiki page fails adequately to distinguish between the value of a wave function and its expected amplitude. I do not deny that conceptualizing of such things as negative probabilities has merits (I think of distributions in a similar way, as mixtures of positive and negative ones): but we are forewarned that this automatically implies our conceptualization violates the axioms of probability. Thus, any rigorous deductions therefrom would have to reprove all the theorems on which they rely. $\endgroup$
    – whuber
    Commented Nov 12, 2021 at 18:35

3 Answers 3

5
$\begingroup$

The negative probabilities can have different interpretations.

  • In calculations we may have negative coefficients.

    In the comments Whuber gives an example of an expression for the probability density function of a sum of gamma distributed variables. This density can be seen as a mixture of gamma distribution where some terms are negative.

    A simpler example is the computation of the probability $$P(\text{$A$ or $B$}) = P(\text{$A$}) + P(\text{$B$}) - P(\text{$A$ and $B$})$$ here there is also a negative coefficient and a negative term $ - P(\text{$A$ and $B$})$, but there is no event whose probability is negative. We are just making a computation where we subtract a probability.

  • The events themselves have negative probability in computations. This is what the slit experiment argues. There is the similar formula $P(\text{$A$ or $B$}) = P(\text{$A$}) + P(\text{$B$}) - P(\text{$A$ and $B$})$ but what they compute is $P(\text{$A$ and $B$})$ and they find out that this term is negative.

    There can be different ways how these negative probabilities occur

    • As probabilities of events in a virtual model. Those events are never observed themselves and may not even be real and are only our interpretation of the underlying mathematical mechanics of physics.

      In quantum mechanics these negative probabilities are not paradoxical because they are pseudo-probabilities. The reality –what we observe – is the only place where probability makes sense. Whatever mathematics and physical description is underlying the observations, it is not 'real' and interpretations of probability are irrelevant in that realm.

      For example, in the double slit experiment the negative probability only arrises because we like to think of a non observable event (the photon goes through both slits) and attach a probability to this. But this idea of a photon going through both slits is an imagination and is only how we interpret the unobservable world behind the observable world. It is an atomic/physical interpretation of a subatomic/metaphysical world.

    • A negative probability function can also arrise to simplify computation or because we use approximations that might become negative. In these cases the mathematics is not physical and only happens to be negative due to the use of a model.

The above gives some ways how negative probability may arrise but in all cases it does not mean that there is really some event that has negative probability. The 'negative probability' is a side-effect from using mathematical expression and falsely interpreting terms in those expressions as correct expressions of probability for some real event. The negative may occur because either the expression is not correct (e.g. approximation or simplification) or because the interpretation of the term as an event is not real (e.g. metaphysical interpretations of a quantum mechanical world with actual point particles passing through two slits simultaneously).

$\endgroup$
1
  • $\begingroup$ Thanks for sharing your thoughts Mr. Empiricus :) $\endgroup$ Commented Aug 4, 2022 at 13:53
3
$\begingroup$

Richard Feynman has an interesting paper on this simply titled "Negative Probability." It is an interesting read. Here is the first paragraph:

It is usual to suppose that, since the probabilities of events must be positive, a theory which gives negative numbers for such quantities must be absurd. I should show here how negative probabilities might be interpreted. A negative number, say of apples, seems like an absurdity. A man starting a day with five apples who gives away ten and is given eight during the day has three left. I can calculate this in two steps: 5 - 10 = -5 and -5 + 8 = 3. The final answer is satisfactorily positive and correct although in the intermediate steps of calculation negative numbers appear. In the real situation there must be special limitations of the time in which the various apples are received and given since he never really has a negative number, yet the use of negative numbers as an abstract calculation permits us freedom to do our mathematical calculations in any order simplifying the analysis enormously, and permitting us to disregard inessential details. The idea of negative numbers is an exceedingly fruitful mathematical invention. Today a person who balks at making a calculation in this way is considered backward or ignorant, or to have some kind of a mental block. It is the purpose of this paper to point out that we have a similar strong block against negative probabilities. By discussing a number of examples, I hope to show that they are entirely rational of course, and that their use simplifies calculation and thought in a number of applications in physics.

The whole paper is available here, and has a number of instances of how negative probabilities, or negative probability densities, can be useful. He notes that perhaps the simplest situation where such negative probabilities arise is when working with characteristic functions, where any density is represented as a superposition or "mixture distribution" of sinusoidal functions, which, of course, can take negative values. He then goes from here to talk about the Wigner distribution.

$\endgroup$
1
  • $\begingroup$ Thanks Mike; very interesting, and also knowing the vocab word "Pseudoprobability distribution" (en.wikipedia.org/wiki/Quasiprobability_distribution) is super helpful. Since I wrote this question, I've been bogged down doing practical research. But I hope someday soon to turn my attention back to this! $\endgroup$ Commented Nov 9 at 21:46
2
$\begingroup$

Since posting this question, I came across Holes in Statistics by Andrew Gelman and Yuling Yao. In section 2, the authors speculate about applying quantum probability laws to modeling in psychometrics and other fields and makes it out to be future work. I would therefore answer the question in the negative.

Edit in '24: Another relevant recent work: Nick Polson and Vadim Solokov Negative Probability

In addition to those academic articles, these resources are useful for negative probability background:

  1. This CV thread.
  2. The Signed Measure Wiki page.
  3. These course notes, which have the interesting fact that any signed measure may be written as the difference of two positive measures.
$\endgroup$
4
  • 1
    $\begingroup$ There are no quantum probability laws. It's annoying people shove "quantum" into something and make it sound cool. There's no reason why would QM laws apply to macro world. However, if you want to use QM's math apparatus then its essence is amplitudes and all kinds of thing you can do with them using matrix operators. Statisticians are pretty good with linear algebra, but they apply the operators on probability vectors. So, do the same with complex valued amplitudes and Hermitian operators and call it "quantum XYZ" if you wish. Trying to somehow extract probability laws from QM is a masochism $\endgroup$
    – Aksakal
    Commented Nov 16, 2021 at 2:55
  • 1
    $\begingroup$ @Aksakal Could you please read section 2 of that article and suggest a different terminology than "quantum probability laws" to describe their ideas? In particular, the second to last paragraph is what I'm trying to describe. $\endgroup$ Commented Nov 16, 2021 at 13:34
  • 2
    $\begingroup$ I read section 2. It’s lame. These are micro world phenomena which do not extend to macro world. He’s doing what Feynman looked at in 1980s. You don’t use math that makes things more difficult. You use math that makes things easy. In QM is easy to work with amplitudes, you get used to it. It Is just journeymen who want to bring their priest to the church who complain about amplitudes. $\endgroup$
    – Aksakal
    Commented Nov 16, 2021 at 14:04
  • $\begingroup$ @Aksakal Thanks for taking the time to read it. I want to emphasize that there is no need for a direct link between quantum and macro phenomena in order for this to be a useful framework. In their psychometric example, they make no claim that quantum phenomena have an influence on the outcome of the experiment. Rather, they are saying that the way the double slit experiment is modeled could also serve as a model for the psychometric phenomenon of one question being asked interfering with the answer given for a second question. $\endgroup$ Commented Nov 16, 2021 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.