Disagreement between the p-value and the confidence interval in a binomial test This is a question regarding the binomial test in R.
What happens if I get a p-value < 0.05, which would generally cause me to reject the null hypothesis that the population value is equal to a certain proportion (in this case 0.75), but the hypothesised proportion to be tested (i.e. the null hypothesis) still lies within the 95% confidence interval?
Do I still reject the hypothesis of equal proportions?
Here is an example R code:
x <- 31 
n <- 50
binom.test(x, n, p=0.75)

# p-value = 0.04812
# 95 percent confidence interval:
# 0.4717492 0.7534989

 A: This is obviously a border case and the CI and test results are not derived in exactly the same manner (the CI is not an inversion of the test). You might want to look up binomial CIs and note that there are many ways they can be calculated all with pluses and minuses. But none of that gets at your central question of whether you should reject the 0.75 hypothesis here. 
Furthermore, you're treating your p value and CI's both as tests and, because you did that, you can't reject 0.75 because now you have two tests and you should have done alpha correction on them. The CI should be treated as something else but you've clearly conveyed in your question that you're thinking of it as a test. Given that you can now select whichever test you like alpha is not 0.05.
Step back from testing for a moment.
You need to think about your numbers. For some reason you've selected exactly 0.75 as the amount to reject. What if you had an enormous N and got 0.74 and could reject it on both the CI and test? Would you reach the same conclusion you would reach with the 0.62 you have right now? Is there some range close to 0.75 that's just about equivalent to 0.75 in your work or is exactly 0.75 very critical? If there is a range how much of it is captured by your CI? How believable is your test rejection then? And what about the range of the CI you do have? It's about 0.25 and that's quite a lot of the possible range of proportions. Do you think you think you can say much about what the true proportion really is? It could be some value really close to 0.75 or it could be close to 0.50. How strong a statement do you want to make with the data you have? Also, do values above 0.75 matter the same as values below? Was 0.75 a lower limit you were testing? In this case the conclusion might be different.
So that's a lot of questions but I put them out there to make a point. Simply rejecting the null is going to be a rather pointless endeavour with these data. Assuming you can make a case to reject it, what else can you say? Is it really useful to tell people the true value is not 0.75 but it could be 0.74?
Collect more data.
A: The problem with tests of binomial proportion is that the tests used are generally approximate (since the exact "Clopper-Pearson" test is ridiculously conservative). Therefore, it's not clear that the procedure used to get the CI is the same as that used to test the hypothesis. Theoretically, either approach should lead to the same conclusion if you are using just one CI and one test. 
You have a border case. Either statistic is telling you that your observation is not all that common under the null hypothesis. Remember: there is nothing special about 5% significance...its a cultural artifact from Ronald Fisher in the 1930's. Its a guideline. 
For what it's worth, I'd conclude that its unlikely that the true success probability is as high as 0.75. 
Per @John
In a strict hypothesis testing situation, you're stuck with your 0.05, so you would reject the null hypothesis under that criteria. However, I wouldn't run to the press quite yet ;-)...hypothesis tests can really destroy any nuance in inference.
A: This is my first time answering a question, so I'm hoping that I am actually providing a useful answer.
When you run this in R:
x <- 31
n <- 50
p <- 0.75
binom.test(x, n, p = p)

... it returns the following results:
    Exact binomial test

data:  x and n
number of successes = 31, number of trials = 50, p-value = 0.04812
alternative hypothesis: true probability of success is not equal to 0.75
95 percent confidence interval: 
 0.4717492 0.7534989
sample estimates: 
probability of success 
                  0.62 

What the p-value of 0.4812 is telling you is that testing for a probability of success of 0.75 (75%) of having 31 "successful" outcomes out of 50 attempts of a binary event (0, 1) -- i.e.: flipping a coin and getting heads up 31 times out of 50 --  is just inside the 95% confidence interval, which are a range of probabilities of success.  
Therefore, you could cautiously accept the null hypothesis that the probability of success is equal to 75%.  (The alternative hypothesis being that the probability of success is not equal to 75%.)
The calculated probability of success is included at the bottom of the output: 0.62, or 62% chance of success.  This is nothing more than 31 / 50.
A: While the test and confidence interval provided by binom.test() are both exact, the confidence interval is unfortunately not based on inverting the test, so they may lead to inconsistent results. See the paper
Fay, M.P. (2010). Two-sided Exact Tests and Matching Confidence Intervals for Discrete Data. R Journal, volume 2, no. 1, pages 53–58.
for more information. Luckily, there is an R package by the author of the above paper that does provide (three different pairs of) consistent tests and confidence intervals. Applied to your data, the results are [output lightly edited to only show the relevant information]:
> library(exactci)
> binom.exact(31, 50, p=0.75, tsmethod="central")
Exact two-sided binomial test (central method)

p-value = 0.05747
95 percent confidence interval:
0.4717492 0.7534989

> binom.exact(31, 50, p=0.75, tsmethod="minlike")
Exact two-sided binomial test (sum of minimum likelihood method)

p-value = 0.04812
95 percent confidence interval:
0.4799 0.7463

> binom.exact(31, 50, p=0.75, tsmethod="blaker")
Exact two-sided binomial test (Blaker's method)

p-value = 0.04812
95 percent confidence interval:
0.4797 0.7463

Consult the above paper for information on the pros and cons of the three different sets of tests and confidence intervals.
