I am trying to build a model with binary output and when used Logistic regression, I found none of the 20 variables(with about 5 factor variables) as significant (p-value higher than alpha=0.05). But when I used the same dataset for XGBoost Model it gave 90% accuracy on test set in R.
Is this normal?. Am I doing right?. I have only 40 observations, and split the training and test sets in the ratio of 75:25%. Does this is any way issue for this difference in the results between 2 models.
What can be done further to have a robust model?. I can think of getting more sample data only.
Any other ideas?