The simple version is that any two variables that tend to change in one direction over time will appear to be correlated, whether there's any connection between them or not. Consider the following variables:
time = seq(from=1, to=100, by=1)
x = .5 + .3*time + rnorm(100)
y1 = 3 + .3*time + rnorm(100)
y2 = 7 + .1*time + .8*x + rnorm(100)
$x$ is just a function of time, as is $y1$. $y2$ is a function of both time and $x$. The point is to recognize from the code that there really is a relationship between $x$ and $y2$, and that there is no relationship between $x$ and $y1$. Now look at the following figure, all three lines look awfully similar, don't they?
In fact, the $R^2$ value for the relationship between $x$ and $y1$ is 98%, and the $R^2$ for $x$ and $y2$ is 99%. But we know that there's no real relationship between $x$ and $y1$, whereas there is between $x$ and $y2$, so how do we differentiate the real from mere appearance? That's where differencing comes in. For any two of the variables, since they both tend to go up over time, that's not very informative, but given that one goes up by some specific amount, does that tell us how much the other goes up? Differencing allows us to answer that question. Note the following two figures, scatterplots I made after differencing all three variables.
Here, we clearly see that knowing something about how much $x$ went up tells us something about how much $y2$ goes up ($R^2=.43$), but that this is not the case for $x$ and $y1$ ($R^2=.07$). So the answer to your question is that you should ignore the correlations amongst your original variables and look at the differenced variables. Given that your $R^2$ is .004, I would say there's no actual relationship.
Some other points: In the figures, I make a point of noting that these are simultaneous changes. There's nothing wrong with that, and it follows from the way I set up the problem, but usually people are interested in effects at some lag. (That is, change in one thing at one point in time leads to change in something else later.) Second, you mention taking the log of one of your series. Taking the log simply switches your data from levels to rates. And thus, when you difference, you are looking at changes in rates rather than changes in levels. That's very common, but I didn't include that element in my demonstration; it's orthogonal to the issues I discussed. Lastly, I want to acknowledge that time series data are often more complicated than my demonstration lets on. A comprehensive overview would require a book length treatment, but @Charlie's answer does a good job of succinctly pointing out some of the complexities that I left out.