Thanks for the help from @whuber. Here is the answer to my own question. The original statement is not true. It should be rephrased to this,

$$ For\ |\mu2-\mu1| < 2\sigma\\ p''(x) < 0, if \begin{cases}\sigma < x < (\mu2 - \mu1) + \sigma\\-\sigma < x < (\mu2 - \mu1) - \sigma\end{cases} $$

Proof:

After some calculation, the second derivative is

$$ p''(x) = \frac{-1}{2\sqrt{2\pi}\sigma^3}[(1 - \frac{(x - \mu1)^2}{\sigma^2}) \exp^{\frac{-(x - \mu1)^2}{2\sigma^2}} + (1 - \frac{(x - \mu2)^2}{\sigma^2}) \exp^{\frac{-(x - \mu2)^2}{2\sigma^2}}] $$

Set $y = \frac{1}{\sigma}(x - \frac{\mu1 + \mu2}{2})$

$$ p''(y) = \frac{-1}{2\sqrt{2\pi}\sigma^3}[(1 - (y + (\frac{\mu2 - \mu1}{2\sigma}))^2) \exp^{\frac{-1}{2}(y + \frac{\mu2 - \mu1}{2\sigma})^2} + (1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2) \exp^{\frac{-1}{2}(y - (\frac{\mu2 - \mu1}{2\sigma}))^2}] $$

By removing the constant, then the objective function is

$$ ((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) \exp^{-y(\frac{\mu2 - \mu1}{\sigma})} + ((y - (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) < 0 $$

We can organize this function into

$$ \exp^{-y(\frac{\mu2 - \mu1}{\sigma})} < \frac{(1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2)}{((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1)} $$

Because the exponential function must be positive, we should try to prove that

$$ (1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2)((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) > 0 $$

Because the 4 roots in this function are

$$ y = \frac{\mu2 - \mu1}{2\sigma}\pm1, -\frac{\mu2 - \mu1}{2\sigma}\pm1 $$

Finally, we can find the interval of y to make this function larger than 0

$$ \begin{alignedat}{2} -\frac{\mu2 - \mu1}{2\sigma} - 1 < y < \frac{\mu2 - \mu1}{2\sigma} - 1\\ 1 - \frac{\mu2 - \mu1}{2\sigma} < y < \frac{\mu2 - \mu1}{2\sigma} + 1 \end{alignedat} $$

After substituing y with $\frac{1}{\sigma}(x - \frac{\mu1 + \mu2}{2})$, we can get

$$ \begin{alignedat}{2} \sigma < x < (\mu2 - \mu1) + \sigma\\-\sigma < x < (\mu2 - \mu1) - \sigma\ \end{alignedat} $$

Thanks for the help from @whuber. Here is the answer to my own question. The original statement is not true. It should be rephrased to this,

$$ For\ |\mu2-\mu1| < 2\sigma\\ p''(x) < 0, if \begin{cases}\sigma < x < (\mu2 - \mu1) + \sigma\\-\sigma < x < (\mu2 - \mu1) - \sigma\end{cases} $$

Proof:

After some calculation, the second derivative is

$$ p''(x) = \frac{-1}{2\sqrt{2\pi}\sigma^3}[(1 - \frac{(x - \mu1)^2}{\sigma^2}) \exp^{\frac{-(x - \mu1)^2}{2\sigma^2}} + (1 - \frac{(x - \mu2)^2}{\sigma^2}) \exp^{\frac{-(x - \mu2)^2}{2\sigma^2}}] $$

Set $y = \frac{1}{\sigma}(x - \frac{\mu1 + \mu2}{2})$

$$ p''(y) = \frac{-1}{2\sqrt{2\pi}\sigma^3}[(1 - (y + (\frac{\mu2 - \mu1}{2\sigma}))^2) \exp^{\frac{-1}{2}(y + \frac{\mu2 - \mu1}{2\sigma})^2} + (1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2) \exp^{\frac{-1}{2}(y - (\frac{\mu2 - \mu1}{2\sigma}))^2}] $$

By removing the constant, then the objective function is

$$ ((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) \exp^{-y(\frac{\mu2 - \mu1}{\sigma})} + ((y - (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) < 0 $$

We can organize this function into

$$ \exp^{-y(\frac{\mu2 - \mu1}{\sigma})} < \frac{(1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2)}{((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1)} $$

Because the exponential function must be positive, we should try to prove that

$$ (1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2)((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) > 0 $$

Because the 4 roots in this function are

$$ y = \frac{\mu2 - \mu1}{2\sigma}\pm1, -\frac{\mu2 - \mu1}{2\sigma}\pm1 $$

Finally, we can find the interval of y to make this function larger than 0

$$ \begin{alignedat}{2} -\frac{\mu2 - \mu1}{2\sigma} - 1 < y < \frac{\mu2 - \mu1}{2\sigma} - 1\\ 1 - \frac{\mu2 - \mu1}{2\sigma} < y < \frac{\mu2 - \mu1}{2\sigma} + 1 \end{alignedat} $$

After substituing y with $\frac{1}{\sigma}(x - \frac{\mu1 + \mu2}{2})$, we can get

$$ \begin{alignedat}{2} \mu1 - \sigma < x < \mu2 - \sigma\\ \sigma - \mu1 < x < \mu2 + \sigma\ \end{alignedat} $$

Thanks for the help from @whuber. Here is the answer to my own question. The original statement is not true. It should be rephrased to this,

$$ For\ |\mu2-\mu1| < 2\sigma\\ p''(x) < 0, if \begin{cases}\sigma < x < (\mu2 - \mu1) + \sigma\\-\sigma < x < (\mu2 - \mu1) - \sigma\end{cases} $$

Proof:

After some calculation, the second derivative is

$$ p''(x) = \frac{-1}{2\sqrt{2\pi}\sigma^3}[(1 - \frac{(x - \mu1)^2}{\sigma^2}) \exp^{\frac{-(x - \mu1)^2}{2\sigma^2}} + (1 - \frac{(x - \mu2)^2}{\sigma^2}) \exp^{\frac{-(x - \mu2)^2}{2\sigma^2}}] $$

Set $y = \frac{1}{\sigma}(x - \frac{\mu1 + \mu2}{2})$

$$ p''(y) = \frac{-1}{2\sqrt{2\pi}\sigma^3}[(1 - (y + (\frac{\mu2 - \mu1}{2\sigma}))^2) \exp^{\frac{-1}{2}(y + \frac{\mu2 - \mu1}{2\sigma})^2} + (1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2) \exp^{\frac{-1}{2}(y - (\frac{\mu2 - \mu1}{2\sigma}))^2}] $$

By removing the constant, then the objective function is

$$ ((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) \exp^{-y(\frac{\mu2 - \mu1}{\sigma})} + ((y - (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) < 0 $$

We can organize this function into

$$ \exp^{-y(\frac{\mu2 - \mu1}{\sigma})} < \frac{(1 - (y + (\frac{\mu2 - \mu1}{2\sigma}))^2)}{((y - (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1)} $$

Because the exponential function must be positive, we should try to prove that

$$ (1 - (y + (\frac{\mu2 - \mu1}{2\sigma}))^2)((y - (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) > 0 $$

Because the 4 roots in this function are

$$ y = \frac{\mu2 - \mu1}{2\sigma}\pm1, -\frac{\mu2 - \mu1}{2\sigma}\pm1 $$

Finally, we can find the interval of y to make this function larger than 0

$$ \begin{alignedat}{2} -\frac{\mu2 - \mu1}{2\sigma} - 1 < y < \frac{\mu2 - \mu1}{2\sigma} - 1\\ 1 - \frac{\mu2 - \mu1}{2\sigma} < y < \frac{\mu2 - \mu1}{2\sigma} + 1 \end{alignedat} $$

After substituing y with $\frac{1}{\sigma}(x - \frac{\mu1 + \mu2}{2})$, we can get

$$ \begin{alignedat}{2} \sigma < x < (\mu2 - \mu1) + \sigma\\-\sigma < x < (\mu2 - \mu1) - \sigma\ \end{alignedat} $$

Thanks for the help from @whuber. Here is the answer to my own question. The original statement is not true. It should be rephrased to this,

$$ For\ |\mu2-\mu1| < 2\sigma\\ p''(x) < 0, if \begin{cases}\sigma < x < (\mu2 - \mu1) + \sigma\\-\sigma < x < (\mu2 - \mu1) - \sigma\end{cases} $$

Proof:

After some calculation, the second derivative is

$$ p''(x) = \frac{-1}{2\sqrt{2\pi}\sigma^3}[(1 - \frac{(x - \mu1)^2}{\sigma^2}) \exp^{\frac{-(x - \mu1)^2}{2\sigma^2}} + (1 - \frac{(x - \mu2)^2}{\sigma^2}) \exp^{\frac{-(x - \mu2)^2}{2\sigma^2}}] $$

Set $y = \frac{1}{\sigma}(x - \frac{\mu1 + \mu2}{2})$

$$ p''(y) = \frac{-1}{2\sqrt{2\pi}\sigma^3}[(1 - (y + (\frac{\mu2 - \mu1}{2\sigma}))^2) \exp^{\frac{-1}{2}(y + \frac{\mu2 - \mu1}{2\sigma})^2} + (1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2) \exp^{\frac{-1}{2}(y - (\frac{\mu2 - \mu1}{2\sigma}))^2}] $$

By removing the constant, then the objective function is

$$ ((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) \exp^{-y(\frac{\mu2 - \mu1}{\sigma})} + ((y - (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) < 0 $$

We can organize this function into

$$ \exp^{-y(\frac{\mu2 - \mu1}{\sigma})} < \frac{(1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2)}{((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1)} $$

Because the exponential function must be positive, we should try to prove that

$$ (1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2)((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) > 0 $$

Because the 4 roots in this function are

$$ y = \frac{\mu2 - \mu1}{2\sigma}\pm1, -\frac{\mu2 - \mu1}{2\sigma}\pm1 $$

Finally, we can find the interval of y to make this function larger than 0

$$ \begin{alignedat}{2} -\frac{\mu2 - \mu1}{2\sigma} - 1 < y < \frac{\mu2 - \mu1}{2\sigma} - 1\\ 1 - \frac{\mu2 - \mu1}{2\sigma} < y < \frac{\mu2 - \mu1}{2\sigma} + 1 \end{alignedat} $$

After substituing y with $\frac{1}{\sigma}(x - \frac{\mu1 + \mu2}{2})$, we can get

$$ \begin{alignedat}{2} \sigma < x < (\mu2 - \mu1) + \sigma\\-\sigma < x < (\mu2 - \mu1) - \sigma\ \end{alignedat} $$