Is it fine to slightly overfit, if its giving you good predictive power? I have 2 variables which have a correlation of about 0.51. In the test dataset the estimate of one is changing from +ve to -ve. But keeping both of them in model is giving me good predictive power. Is it fine to keep both of them in model? or it might to lead to some wrong results?
 A: Sounds like your issue is collinearity rather than overfitting, as user1320502 suggested in a comment.
Do you know where these two variables came from? For example, if one is $x$ and one is $x^2$, centering the variables may help.
If all you care about is predicting, collinearity is not a direct problem. But if you look at other things (confidence intervals, etc), collinearity will affect things.
You might look at similar questions like:
How to solve collinearity problems in OLS regression? 
How to prevent collinearity? (the question is about preventing collinearity, but see Aleksandr Blekh's answer which talks about dealing with collinearity if you didn't prevent it in the first place.
Importance of multiple linear regression assumptions when building predictive regression models (see the OP's comment on Michael Bishop's answer: nested CV)
A: How do you know that your model is overfitted? If an "overfitted" model (let us call it model A) is giving you truly better predictive power (no cheating, honest out of sample assessment) than some benchmark model that you think is non-overfitted (call it model B), I would suspect that model B is actually underfitted while model A is 


*

*less-underfitted than model B or 

*non-overfitted or perhaps 

*slightly overfitted (but not as severely as model B is underfitted).


So I would say it is fine to use model A in place of model B if you have to choose one of the two. 
Regarding whether to keep both predictors or drop one, I would suggest making the choice based on out-of-sample performance assessment. If a model containing both of them gives better forecasts, choose it.
