I understand that the normal distribution is undefined if the standard deviation is zero, but I need to handle the case where all values are equal in a computer algorithm. The following method must return a valid value, even if the standard deviation is zero. How can I fix this method so it does not divide by zero?

public static double NormalDist(double x, double mean, double standard_dev)
   double fact = standard_dev * Math.Sqrt(2.0 * Math.PI);
   double expo = (x - mean) * (x - mean) / (2.0 * standard_dev * standard_dev);
   return Math.Exp(-expo) / fact;

My idea was to insert this at the beginning of the method:

        if (standard_dev == 0.0)
            return x == mean ? 1.0 : 0.0;

Would this be correct?

  • 1
    $\begingroup$ You didn't "fix" it.The PDF of normal distribution is undefined for $\sigma=0$, it's not 1 like in your "fix" $\endgroup$ – Aksakal Sep 7 '16 at 16:54
  • 1
    $\begingroup$ @Aksakal points to an important issue: exactly how do you intend to use NormalDist in your algorithm? $\endgroup$ – whuber Sep 7 '16 at 17:05
  • $\begingroup$ This is being used in a naïve bayes classifer. It's attempting to select a class based on data in a training set. To me, if seems that if all values in a set of parameters are the same, they should have no influence on the outcome. So, for example, if we are looking at how shoe size falls within a distribution to determine if a person (the class) is male or female, but if it just so happens that all persons in the distribution have exactly the same shoe size, then shoe size cannot be used to differentiate and should be removed from consideration. I am looking to handle this case. $\endgroup$ – user130307 Sep 7 '16 at 18:32
  • 1
    $\begingroup$ Maybe the answer is not in changing the output of this method, but rather to just throw out that value altogether. $\endgroup$ – user130307 Sep 7 '16 at 18:36

When standard deviation is zero, your Gaussian (normal) PDF turns into Dirac delta function. You can't simply plug zero standard deviation into the conventional expression. For instance, if the PDF is plugged into some kind of numerical integration, this won't work. You have to modify the integrals. In the example below we calculate the mean value of function $g(x)$ using the Gaussian density $f(x|\mu,\sigma^2)$:

$$\int g(x)f(x|\mu,\sigma^2)dx$$

when you plug zero variance, this becomes delta-functional: $$\int g(x)f(x|\mu,0)dx=\int g(x)\delta(x-\mu)dx=g(\mu)$$

Your code has to be able to recognize this, otherwise it'll fail.

One way to fix this is surprisingly simple: plug a very small value of $\sigma$ into Gaussian instead of zero. You'll have to pick the right $\sigma$ for your situation. If it's too small then it'll blow up your exponent, and the integrals will not work or the precision will be low. This goes to a known Gaussian approximation of delta function: $$\delta(x)=\lim_{\sigma\to 0}\mathcal N(0,\sigma)$$

  • $\begingroup$ Whether it "fails" has to depend on its functional specification. If the intention is merely to return a continuous version of the probability density, then the function that always returns zero is correct, because the Dirac delta has a density defined everywhere but at the mean. If, on the other hand, this function is being called as part of a likelihood calculation for computing an ML estimate, then there is no implementation whatsoever that will cause it to succeed: all it can do is raise an error condition. $\endgroup$ – whuber Sep 7 '16 at 18:25
  • $\begingroup$ @whuber, floating point format supports positive infinity. So, he could set the PDF to infinity. The problem's that even then his likelihood optimization routine will not work, because of the discontinuity. Hence, I'm saying that he has to modify the code that uses PDF. He can't get away with just messing with PDF alone $\endgroup$ – Aksakal Sep 7 '16 at 18:42
  • $\begingroup$ I posted an additional comment above with more info. I'm thinking Aksakal is correct. I have to modify the code that uses the method. $\endgroup$ – user130307 Sep 7 '16 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.