Yes, you can. A dummy variable is no different, mathematically, from any other fixed effect you might choose to include. Presumably, if some of the categories all have the same impact on the response, it would make sense to zero them out and push their effect into the intercept term. 

That said, using Wald statistics to cull variables is risky. You may get the right set of variables, but this isn't necessarily the case. 

When you say "random slope", are you talking about the coefficient of the categorical variable? If so, I would do some model checking. Look at the estimated random effects and see if they are trying to cover small but real differences in your categories.