How to get the derivative of a normal distribution w.r.t its parameters? We normally calculate the derivative of normal density w.r.t its parameters, mean and variance. But can we calculate the derivative of normal distribution wrt the parameters(not the variable, I know the derivative wrt to the variable gives the density)? If yes, how do we calculate that?
 A: Just apply the chain rule for differentiation. The CDF 
$F_X(x; \mu, \sigma^2)$ of a $N(\mu,\sigma^2)$ random variable $X$
is $\Phi\left(\frac{x-\mu}{\sigma}\right)$ and so 
$$\frac{\partial}{\partial \mu}F_X(x; \mu, \sigma^2)
=\frac{\partial}{\partial \mu}\Phi\left(\frac{x-\mu}{\sigma}\right)
= \phi\left(\frac{x-\mu}{\sigma}\right)\frac{-1}{\sigma}
= -\left[\frac{1}{\sigma}\phi\left(\frac{x-\mu}{\sigma}\right)\right]$$
where $\phi(x)$ is the standard normal density and the quantity in
square brackets on the rightmost expression above can be recognized
as the density of $X\sim N(\mu,\sigma^2)$.
I will leave the calculation of the derivative with respect 
to $\sigma$ or $\sigma^2$ for you to work out for yourself.
A: It's a simple calculus. Remember that an integral (which is the cumulative probability function) is basically a sum. So, a derivative of a sum is the same as a sum of derivatives. Hence, you simply differentiate the function (i.e. density) under the integral, and integrate. This was my bastardized version of the fundamental theorem of calculus, that some didn't like here.
Here's how you'd do it with the normal probability. First, the general relation for probability function $F(x;\mu,\sigma)$ and the density $f(x;\mu,\sigma)$ where the mean and the standard deviation are the parameters:
$$\frac{\partial}{\partial \mu} F(x;\mu,\sigma)=\frac{\partial}{\partial \mu}\int_{-\infty}^x f(x;\mu,\sigma) dx=\int_{-\infty}^x \frac{\partial}{\partial \mu} f(x;\mu,\sigma) dx$$
You, actually, used a more general form of this manipulation called Leibnitz rule when you mentioned that the differentiation of the probability function by the variable itself (i.e. $\frac{\partial}{\partial x}$) will give you the density (PDF).
Next, plug the density:
$$=\int_{-\infty}^x \frac{\partial}{\partial \mu} \frac{e^{-(x-\mu)^2/\sigma^2}}{\sqrt{2\pi}\sigma} dx=\int_{-\infty}^x\frac{2(x-\mu)}{\sigma^2} \frac{e^{-(x-\mu)^2/\sigma^2}}{\sqrt{2\pi}\sigma} dx$$
Change of variables $\xi=\frac{(x-\mu)^2}{\sigma^2}$:
$$=\frac{1}{\sqrt{2\pi}\sigma}\left(-\int_0^\infty e^{-\xi} d\xi + \int_{0}^{\xi(x)} e^{-\xi}d\xi\right)
=\frac{1}{\sqrt{2\pi}\sigma}\left( -1-(e^{-\xi(x)}-1) \right)$$
$$=-\frac{e^{-\frac{(x-\mu)^2}{\sigma^2}}}{\sqrt{2\pi}\sigma}$$
Hence, you have the following:
$$\frac{\partial}{\partial \mu} F(x;\mu,\sigma)=-f(x;\mu,\sigma)$$
You can a similar trick with the variance.
