Can I trust a model if I do not check the prediction error? My main field is machine learning and 90 % of what I do is to try to improve the prediction error. 
Recently I have started to work with a medical group. They are mainly doctors so I do not know how much I can trust their statistical knowledge. 
What they commonly do is to fit logistic regression and without any consideration of the model performance they start to look to the coefficient in order to understand which one is more important or stuff like that. 
In a similar way I fitted a random forest model and the I have done the importance plot something like this even if the predicted performance was very poor. 
How much can I trust these kind of results in model where the prediction performance is poor? 
 A: Depends on what you are looking for.
First of all, we have to define what "trusting a model" is. From a mathematical point of view, this mean : "are you using the correct model according to your data ?". In this case, you should not be using the prediction error to assess the quality of your fit. The only thing we can do is "checking that we have no evidence that the assumptions we made for this model are violated". In the case of linear regression, we usually check the normality of the error distribution, linearity of the relationship between covariates and response variable, homoscedasticity and the independence of errors.
As Glen_b said, models have more than one possible purpose. If you think about prediction performance, I'm pretty sure we can find an example where they are good but the model is not. This case could be very problematic if you think that you are using the correct model. This could lead to wrong prediction for data that are far away from your original data.
There are many other things you have to be careful about (i.e. number of observations, variance of your observations, etc).
