Problems with taking the first difference of a stationary series Suppose you have sufficient observations for running a time-series regression, but you do not have sufficient observations to accurately test for the stationarity of the residuals. If you do take the first difference (as a conservative approach), you have the possibility of have taken the first difference in a model in which the residuals might be stationary, although you do not have the data to confidently test for that. What are the problems that you are risking to meet with this approach?
 A: Lets say your time series $x_t$ is stationary, as you say. You didn't say what type of stationarity so I'll assume each $x_t$ represents an iid observation from some distribution.
What can happen, then, if you form a series $d_t=x_{t+1}-x_t$? Right off the bat, its highly likely that your resulting distribution will change (unless they are normally distributed), but worse, you run the risk of inducing an artificial autocorrelation structure in your time series: Large differences tend to be followed by equally large differences in the opposite direction (so as to preserve the stationarity of the undifferenced time series). Finally, the variance of the time series will actually increase: $\operatorname{Var}(x_{t+1}-x_t)=2\operatorname{Var}(x_{t+1})$, since the $x_i$ are iid. Had your observations actually been positively correlated, then it's likely this would decrease for strong correlations.
However, you should be able to check the correlogram of your time series to identify any autocorrelation and/or trend. At least check the autocorrelation statistics before and after differencing to see.
