Note that the derivative is evaluated at $0$ in a limiting sense.

That is, the $n$th order moment of $X$ is given by

$$E[X^n]=M^{(n)}(t)\Big\lvert_{t=0}=\lim_{t\to 0}M^{(n)}(t)$$

As you say, the derivatives of $M(t)$ are not defined at $t=0$.

For $t\ne 0$, the first derivative for example is $$M'(t)=\frac{1}{t^2(b-a)}\left[e^{tb}(tb-1)-e^{ta}(ta-1)\right]$$

But note that $M'(t)\to \frac{a+b}{2}$ as $t\to 0$, so $M'(t)$ has a removable discontinuity at $t=0$.

So just like $M(t)$ itself, we define

$$M'(t)=\begin{cases} \frac{1}{t^2(b-a)}\left[e^{tb}(tb-1)-e^{ta}(ta-1)\right]&,\text{ if }t\ne 0 \\\frac{a+b}{2}&,\text{ if }t=0 \end{cases}$$

Hence the first moment is given by $$E[X]=M'(0)=\lim_{t\to 0}M'(t)$$

If you do this from definition, you will end up with the same result:

$$M'(0)=\lim_{t\to 0}\frac1t \left[M(t)-M(0)\right]=\frac{a+b}{2}$$

We define the $r$th order derivative $M^{(r)}(t)$ similarly so that it is continuous at $0$.

And the $r$th order moment of $X$ for $r\in \mathbb N$ is given by

$$E[X^r]=M^{(r)}(0)=\lim_{t\to 0}M^{(r)}(t)$$