Thanks for the help from @whuber. Here is the answer to my own question.
The original statement is not true. It should be rephrased to this,

$$
For\ |\mu2-\mu1| < 2\sigma\\
p''(x) < 0, if \begin{cases}\sigma < x < (\mu2 - \mu1) + \sigma\\-\sigma < x < (\mu2 - \mu1) - \sigma\end{cases}
$$

Proof:

After some calculation, the second derivative is 

$$
p''(x) = \frac{-1}{2\sqrt{2\pi}\sigma^3}[(1 - \frac{(x - \mu1)^2}{\sigma^2}) \exp^{\frac{-(x - \mu1)^2}{2\sigma^2}} + (1 - \frac{(x - \mu2)^2}{\sigma^2}) \exp^{\frac{-(x - \mu2)^2}{2\sigma^2}}]
$$

Set $y = \frac{1}{\sigma}(x - \frac{\mu1 + \mu2}{2})$

$$
p''(y) = \frac{-1}{2\sqrt{2\pi}\sigma^3}[(1 - (y + (\frac{\mu2 - \mu1}{2\sigma}))^2) \exp^{\frac{-1}{2}(y + \frac{\mu2 - \mu1}{2\sigma})^2} + (1 - (y - (\frac{\mu2 - \mu1}{2\sigma}))^2) \exp^{\frac{-1}{2}(y - (\frac{\mu2 - \mu1}{2\sigma}))^2}]
$$

By removing the constant, then the objective function is

$$
((y + (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) \exp^{-y(\frac{\mu2 - \mu1}{\sigma})} + ((y - (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) < 0
$$

We can organize this function into

$$
\exp^{-y(\frac{\mu2 - \mu1}{\sigma})} < \frac{(1 - (y + (\frac{\mu2 - \mu1}{2\sigma}))^2)}{((y - (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1)}
$$

Because the exponential function must be positive, we should try to prove that

$$
(1 - (y + (\frac{\mu2 - \mu1}{2\sigma}))^2)((y - (\frac{\mu2 - \mu1}{2\sigma}))^2 - 1) > 0
$$

Because the 4 roots in this function are

$$
y = \frac{\mu2 - \mu1}{2\sigma}\pm1, -\frac{\mu2 - \mu1}{2\sigma}\pm1
$$

Finally, we can find the interval of y to make this function larger than 0

$$
\begin{alignedat}{2}
-\frac{\mu2 - \mu1}{2\sigma} - 1 < y < \frac{\mu2 - \mu1}{2\sigma} - 1\\
1 - \frac{\mu2 - \mu1}{2\sigma} < y < \frac{\mu2 - \mu1}{2\sigma} + 1
\end{alignedat}
$$

After substituing y with $\frac{1}{\sigma}(x - \frac{\mu1 + \mu2}{2})$, we can get

$$
\begin{alignedat}{2}
\sigma < x < (\mu2 - \mu1) + \sigma\\-\sigma < x < (\mu2 - \mu1) - \sigma\
\end{alignedat}
$$