In statistical software like MINITAB and SAS, the default alpha value (critical p-value) is set as 0.15. I would like to know (1) if there is any statistic basis to set it as 0.15 and (2) if this is a common accepted value in statistics.
I have found that in this question, the critical p-value of 0.15 in R is actually based on AIC. So is AIC the basis to set 0.15 as default value?
Thanks in advance for your help!
The basis is cargo cult science. "$p$-values" produced from stepwise regression model building do not have the typical meaning of "probability of observing the estimate/test statistic assuming the null hypothesis is true," but rather "probability of observing the estimate/test statistic based on a series of unstated conditionals that are almost certainly predicated on some number of variables not included in the presented model."
Use stepwise regression if you want to appear to be performing meaningful statistical analysis, while providing results (estimates, "noise" variables, missing "real" variables, $p$-values, $R^{2}$, etc.) that are very likely to be biased.
Stepwise variable selection causes serious bias when the $\alpha$ level for retaining variables is below 1.0 (and especially if it is below about 0.3). You are not addressing this bias in the $\beta$s. You need to incorporate penalization if using stepwise selection, or use the bootstrap to describe the disaster.
In statistical software like MINITAB and SAS, the default alpha value (critical p-value) is set as 0.15. I would like to know (1) if there is any statistic basis to set it as 0.15
Not really. To my vague recollection (I have no reference to offer), it was originally just presented as a value that seemed to work fairly well, because 0.05 was too strict.
I have found that in this question, the critical p-value of 0.15 in R is actually based on AIC. So is AIC the basis to set 0.15 as default value?
I think the suggestion preceded wide awareness of the AIC (but it's possible that the idea to use it originally came via the AIC, it's nearly old enough). So I think it's more coincidental (though that "a value that seems to work fairly well" would come up close to AIC is hardly surprising).
and (2) if this is a common accepted value in statistics.
It was fairly widely accepted once (many decades ago). The whole idea (no matter what cutoff value is used) is now well understood to be pretty bad.
If I dig up any more precise information, I'll come back and add it.
See p195 of "Clinical Prediction Models" by EW Steyerberg. To select a candidate predictor based on AIC, the chi-squared value has to exceed twice its df. The P-value for a 1 df chi-squared that justifies the model change is 0.157 (which is where I think the 0.15 comes from). However the p value for a 2 df model (eg if you wanted to see if a nominal variable with 3 categories should be added) is 0.135. Other words the 0.15 criterion only applies to 1df changes. Stepwise selection of models using AIC adapts the critical p-value in an implicit way and using higher p-values allows inclusion of predictors with weaker effects, although the best results use model averaging. The AIC is also asymptotically equivalent to cross-validation
