What is the meaning of a large p-value? I understand that the $p$-value is the conditional probability of observing the test statistic or something more extreme given that the null hypothesis is true. I have read the great explanation by @user28 in this post: What is the meaning of p values and t values in statistical tests? However, do large $p$-values say anything? Does a larger $p$-value lend greater support to the null hypothesis? If I set rejection region to be $<0.05$, then does it make a difference if I get $p$-value $0.06$ or $0.99$? (After all, $0.05$ is arbitrary, and $0.06$ is so close to being rejected that if I arbitrarily set $0.05$ as $0.1$ instead, the null hypothesis would have been rejected.) Can one make any statistical use of a non-rejecting $p$-value?
 A: How you should 'use' the p-value depends on how you have designed your study with regard to the analyses you will run.  I discuss two different philosophies about p-values in my answer here: When to use Fisher and Neyman-Pearson framework?  You may find it helpful to read that.  If you have, for example, run a power analysis and intend to use the p-value to make a final decision, you should not use close to the line ('marginally significant') as a meaningful category.  It is fine to use a different alpha than $0.05$ (such as $0.10$), but once you decided on it and set your study up accordingly, you should stick with it.  
In addition, you cannot use a large p-value as evidence for the null hypothesis.  I discussed that idea in my answer here: Why do statisticians say a non-significant result means "you cannot reject the null" as opposed to accepting the null hypothesis?  Reading that answer may be helpful to you as well.  
A: In my view, everything boils down to assumptions, that is, how well the models fits them. If it does agree with all of them, treat the p-value as a probability. Then you can compare p-value of 0.06 with 0.99 by concluding which of the two are more likely. Also, a lot depend on circumstances: in some cases, marginal significance shouldn't be ignored, because as you stated, rejection region can be set rather arbitrarily. But if the models satisfies the assumptions, then you should not seek to reject your hypothesis to some arbitrary level but rather investigate, how likely is the outcome that you got. 
A: It is true that the acceptance range for the p-value of a hypothesis test is rather arbitrary, but nevertheless a lower p-value means that the test result can be accepted with more certainty, because the p-value essentially defines the confidence interval for the estimate, so a narrower confidence interval should be regarded more significant for the test.
