Suppose you are using $\alpha = 0.01$ as the significance level. IF you are not getting significant results, is it better to increase the significance level to $\alpha = 0.05$?
Expanding on @EpiGrad 's answer (which is a good answer):
There are many reasons to ignore p-values altogether: Principally, they answer a question we are very rarely interested in.
If you are going to use p-values, using them as cutoffs often makes little sense.
If you are going to use them as cut-offs, you should decide on the cutoff before the analysis
Making a more stringent cutoff for type I error means lower power (more type II error). Typical values are .05 for type I and .20 for type II (power = .8). But there is no reason why type II errors are necessarily less bad than type I errors. Suppose you develop a drug that treats a disease that is terminal and rapidly so (e.g. something like Ebola). You test it.
Type I error - you say the drug does something when it doesn't, and then give a useless drug to dying people.
Type II error - you say the drug does nothing when it does something, and you fail to give a beneficial drug to dying people.
Which is worse? Type II, by my book.
To quote Prof. David Cox
There are no routine statistical questions, only questionable statistical routines
Just if you're not getting significant results? No. Fiddling with the significance level after the experiment is conducted and the results are known is never good practice.
There are circumstances where you might chose a more relaxed p-value, but doing it post hoc is just a bad idea.
On the one hand it is somewhat artificial to discount a variable because its p value is higher than 0.01 (that's an unusually stringent criterion). How you get there may be more important than what is the ultimate significance level. A variable that is well grounded on logic or causal links with an acceptable p value may be much more meaningful than a variable with a lower p value but with no meaningful logic supporting it.
If you are dealing with hypothesis testing, watch out that the statistical significance is in good part just a function of your sample size. A large sample size will translate into a low standard error and higher statistical significance. And, this process is somewhat artificial as large samples will render immaterial differences statistically significant. If you are dealing within such a domain I recommend you move towards an Effect Size method where the unit of statistical distance is not Standard Error but instead Standard Deviation. And, the latter can't be manipulated by sample size.
I found myself asking the same question and came here looking for good arguments, and I have to say I'm not convinced. Let me explain why I think it is fine to change significance level after an experiment.
Whatever you calculate using your chosen significance level a, will have the same value whether you choose it before or after the experiment. In other words, given some results, it is not possible to infer if the choice of a was done before or after.
Ethics only becomes an issue if you are unclear about what you are doing. If you say that you measured a significant effect without specifying what you mean by that, it can be misleading. This is irrelevant, though, if you state, as you should, what a you chose. If your data is available, all the better, since your calculations can be easily reproduced by anyone interested.
If someone decides to use your results, they will have their own criteria to what magnitude of type I and type II errors they are willing to accept. If you get a significant result using a higher a, it can still be an useful information. It is up to whoever is using your information to decide if the chosen a is low enough for them or not.
To illustrate the last argument, I'll draw from the anecdote from @dilip-sarwate. Say a farmer did paint the targets on the side of his barn before and tried to shoot at them. He sees then that the bullets landed outside the targets, but only by a bit. If he decides, afterwards, to increase the radii of the targets, one could call it cheating, but I think it is still valid information: that is, he is not as accurate as he expected to be, but he is still somewhat accurate and one could even say by how much. You then replace an useless result, 'the farmer is not good with a gun', with a more concrete one 'the farmer is accurate to within 5 cm'.