When regressors aren't standardized, their (estimated) coefficients don't tell you much about their importance. A small coefficient like in your example could still reflect a highly predictive variable, if that variable simply happened to have a very wide range (or, equivalently, a small unit). For instance, if I regress housing prices against their floor area in square centimeters, I will probably find a very small slope: a single square centimeter more will not increase the price of a house very much. However, this does not mean that floor space is not an important determinant of housing price. It only gives the slope of the coupling, but not how tight this coupling is. And it's clearly quite an arbitrary number, because if I were to express floor space in square meters rather than square centimeters, the slope would automatically increase by a factor 10,000, just because there are 10,000 square cm in a square meter.
So the size of the coefficient doesn't tell you anything in your example, unless you also have an idea of the variability in that regressor. Fortunately, you also checked whether the inclusion of this regressor improved your fit. The fact that it didn't is a much more important sign than the size of the coefficient, and does indeed suggest that you might as well leave this variable out of your model. (Note that there are more formal ways to test whether a variable improves the fit of a model to a statistically significant degree, but eyeballing the data is a good start.)