Do you reject the null hypothesis when $p < \alpha$ or $p \leq \alpha$?  This is clearly just a matter of definition or convention, and of almost no practical importance. If $\alpha$ is set to its traditional value of 0.05, is a $p$ value of 0.0500000000000... considered to be statistically significant or not? Is the rule to define statistical significance usually considered to be $p < \alpha$ or $p \leq \alpha$?? 
 A: Relying on Lehmann and Romano, Testing Statistical Hypotheses, $\leq$.  Defining $S_1$ as the region of rejection and $\Omega_H$ as the null hypothesis region, loosely speaking, we have the following statement, p. 57 in my copy:

Thus one selects a number $\alpha$ between 0 and 1, called the level
  of significance, and imposes the condition that:
... $P_\theta\{X \in S_1\} \leq \alpha \text{  for all } \theta \in
 \Omega_H$

Since it is possible that $P_\theta\{X \in S_1\} = \alpha$, it follows that you'd reject for p-values $\leq \alpha$.
On a more intuitive level, imagine a test on a discrete parameter space, and a best (most powerful) rejection region with a probability of exactly 0.05 under the null hypothesis.  Assume the next largest (in terms of probability) best rejection region had a probability of 0.001 under the null hypothesis.  It would be kind of difficult to justify, again intuitively speaking, saying that the first region was not equivalent to an "at the 95% level of confidence..." decision but that you had to use the second region to reach the 95% level of confidence.
A: You've touched on an interesting and somewhat controversial issue.  This can be humorously summarized by this image (found on Andrew Gelman's blog but originally courtesy of Dan Goldstein): 

First of all, there is nothing magical about .05.  As long as you pick your threshold beforehand, a threshold of .1 or .01 could make just as much sense.  To that end, either choosing that you want to use a cutoff of $<.05$ or $\leq.05$ would be equally justifiable, provided that you did not cheat by changing your cutoff after having observed your p-value.
If you want to look at this in the strictest sense then, if you beforehand chose a cutoff of $<.05$ (which I believe to be more "standard") and you observe p to be exactly equal to .05, technically you'd be cheating under standard frequentist techniques.  But therein lies part of the problem with this whole approach.  We are making a binary problem "statistically significant or not" out of something that is not really a binary problem at all.  As Andrew Gelman and Hal Stern aptly put it, "The difference between 'significant' and 'not significant' is not itself statistically significant."
