What is the normal distribution when standard deviation is zero? I understand that the normal distribution is undefined if the standard deviation is zero, but I need to handle the case where all values are equal in a computer algorithm. The following method must return a valid value, even if the standard deviation is zero. How can I fix this method so it does not divide by zero?
public static double NormalDist(double x, double mean, double standard_dev)
{
   double fact = standard_dev * Math.Sqrt(2.0 * Math.PI);
   double expo = (x - mean) * (x - mean) / (2.0 * standard_dev * standard_dev);
   return Math.Exp(-expo) / fact;
}

My idea was to insert this at the beginning of the method:
        if (standard_dev == 0.0)
        {
            return x == mean ? 1.0 : 0.0;
        }

Would this be correct?
 A: When standard deviation is zero, your Gaussian (normal) PDF turns into Dirac delta function. You can't simply plug zero standard deviation into the conventional expression. For instance, if the PDF is plugged into some kind of numerical integration, this won't work. You have to modify the integrals. In the example below we calculate the mean value of function $g(x)$ using the Gaussian density $f(x|\mu,\sigma^2)$:
$$\int g(x)f(x|\mu,\sigma^2)dx$$
when you plug zero variance, this becomes delta-functional:
$$\int g(x)f(x|\mu,0)dx=\int g(x)\delta(x-\mu)dx=g(\mu)$$
Your code has to be able to recognize this, otherwise it'll fail.
One way to fix this is surprisingly simple: plug a very small value of $\sigma$ into Gaussian instead of zero. You'll have to pick the right $\sigma$ for your situation. If it's too small then it'll blow up your exponent, and the integrals will not work or the precision will be low. This goes to a known Gaussian approximation of delta function: $$\delta(x)=\lim_{\sigma\to 0}\mathcal N(0,\sigma)$$
A: This is a question in the Statistics textbook by Hogg and Craig!  The authors give a hint:  Look at the moment generating function of the normal and plug in sigma = 0.
So before going to the answer, let's remember why this works - moment generating functions are unique.
The moment generating function of the normal,  N(a,b^2), M(t|a,b^2) =
exp(at + t^2 * b^2 /2)
Setting b=0, we have
M(t|a,B^2) = exp (at)
This is the moment generating function of
f(x) = a.   (I wouldn't call this a Dirac Delta function, I would call this a constant.  Note the Dirac Delta function is not technically a PDF.)
This result shouldn't be a surprise.  As the variance decreases, the probability gets closer to the mean, and so the limiting distribution is the function equals the mean.
Of course this can be proven directly; we can look at a sequence of functions fn(x)  = N(a,b^2/n) and we see the sequence has the variance approach 0 as n approaches infinity. Seeing it converges in probability to the constant is pretty easy; you can show it converges almost surely but this will take a little more work.
But that isn't the exact question - which can be answered as Hogg and Craig suggested!
