Suppose you are using $\alpha = 0.01$ as the significance level. IF you are not getting significant results, is it better to increase the significance level to $\alpha = 0.05$?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
Expanding on @EpiGrad 's answer (which is a good answer):
To quote Prof. David Cox
Just if you're not getting significant results? No. Fiddling with the significance level after the experiment is conducted and the results are known is never good practice.
There are circumstances where you might chose a more relaxed p-value, but doing it post hoc is just a bad idea.
On the one hand it is somewhat artificial to discount a variable because its p value is higher than 0.01 (that's an unusually stringent criterion). How you get there may be more important than what is the ultimate significance level. A variable that is well grounded on logic or causal links with an acceptable p value may be much more meaningful than a variable with a lower p value but with no meaningful logic supporting it.
If you are dealing with hypothesis testing, watch out that the statistical significance is in good part just a function of your sample size. A large sample size will translate into a low standard error and higher statistical significance. And, this process is somewhat artificial as large samples will render immaterial differences statistically significant. If you are dealing within such a domain I recommend you move towards an Effect Size method where the unit of statistical distance is not Standard Error but instead Standard Deviation. And, the latter can't be manipulated by sample size.