I think it depends on your design. If you are doing an exploratory analysis, and building up the model complexity to see if the addition of predictors improves the model fit, then I would use the Likelihood Ratio Test to determine which variables improve the goodness of fit. LRT is quite widely used in model selection, for example:
Baayen, R. H. (2008). Analyzing Linguistic Data. A Practical Introduction to Statistics Using R. Cambridge University Press.
Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015). Parsimonious mixed models. arXiv preprint arXiv,1506.04967.
You can also consider using the information criteria. In general, AIC tends to favour more complex models, whereas BIC penalises heavier for model complexity, so typically favours simpler models.
If you are conducting a confirmatory study, and all of your variables are hypothesised to be significant, then I would favour the more complex model, as finding that a variable is not significant is a finding in itself.
Hope this helps.