Probably not if you only have two covariates. If you had 50 covariates and two were essentially replicates (like in your example), then yes, you could probably eliminate one without risking losing explanatory value of your covariate set.
Run regression in both scenarios. Compare the Multiple R-Squared value (an output of
lm() in R. This will tell you if adding the additional covariate does anything to explain the variance in your response variable.
Also, you could try running regression using training/testing datasets, to better understand the predictive ability of a model with/without the highly collinear covariate. This could give you important insights (i.e. maybe one is better at prediction, but it may also have an extremely heavy tail and occasionally make horrendously erroneous predictions).
If the correlation coefficient is exactly 1 or -1, then delete it.