Is it justified to include non-significant predictors in a model if it decreases the fit criterion (AIC)?

During analysis of a high dimensional dataset (92 cases, 400+ variables) with the goal of statistical inference, I first used a bootstrapped LASSO (bolasso) to select predictors, and then did an OLS/logistic regression on the most often selected predictors. I proceeded by starting from the null model, and included predictors in decreasing order of selection frequency up to the first non-significant predictor.

Now, is it possible to get a better fit (as measured by AIC, for example) by including more (non-significant) predictors? And in that case, should I choose the model with only significant predictors, or the one that has the lowest AIC for best inference?

My understanding of it is that if I include a non-significant predictor in the model, it may yield a better fit only by chance, so it is not justified to include it in the model. Is that correct?

• I don't think it is possible that including an insinificant regressors would reduce AIC. I do not have a formal proof, but I read some place on Cross Validated that AIC is somehow equivalent to a cutoff $p$-value of around 0.15, which is above any of the conventional significance levels. I also read some other place on Cross Validated that significance testing should not be used for variable selection. – Richard Hardy Nov 22 '15 at 20:32
• It’s been five years, but what is your goal in modeling the data? – Dave Jul 4 '20 at 14:59
• "Statistical significance" is a bad goal and the sample size is too small by a factor of 100 for reliably determining which predictors are 'important". And the use if AIC or any method to remove variables up front will destroy the meaning of p-values. – Frank Harrell Dec 5 '20 at 12:02
• @FrankHarrell yes. Unfortunately, I am dealing with medical academics, here. They are not too much concerned about whether the results make sense. – Raoul Dec 6 '20 at 14:19
• You can't let yourself be involved in a project where the investigators are doing bad science, if you can't change them. This project badly needs a heavy dose of unsupervised learning (data reduction masked to Y) to collapse the questions into what the sample size will support. – Frank Harrell Dec 6 '20 at 17:01