What criteria tell us that the prediction of a model is reliable What criteria can be used to tell whether the prediction of a model will be more reliable than other specifications.
Background: 


*

*We have data with $N$ computers.

*However, prices available only for, approx., $N/2$ computers.

*I build bunch of different models using these $N/2$ observations.

*Using one of these model (the "best" one) I want to predict the prices which is not available (they doesn't exist in real) in my data.

*What criteria can be considered to tell whether some model is the "best" in my case if comparison with other specifications?


I am inclined to believe that $R^2_{adj.}$ is an appropriate measure here. Is that right?
Please, look at my answer below.
 A: How well your model works and how well your model fits are different questions.
Have a look at the Wikipedia page on "goodness of fit". $R^2$ is a good start. You should also check properties of the residuals. E.g. quantile plots, residual power, etc. They'll help you understand how errors are distributed and why. That'll help you understand how well your model fits.
I suggest dividing the data you do have prices for, fitting your model with some of it and using the rest to cross-validate: an 80/20 split is conventional. Performance on the validation set can then be evaluated using the same techniques as mentioned above. That'll help you understand how well you model might work, assuming you have enough data and the future instances you're predicting are like the past.
The only real answer to how well your model actually works is to forward test it and evaluate. I.e. try it and see. The true answer is always retrospective.
A: If we want to compare the predictive wellness of models. Here is what I gather from the replies:
Absolute values


*

*AIC / BIC. These criteria can be used when a dependent variable is the same. Moreover, BIC-value is useful only when the number of observations is equal among the candidate model. These are appropriate for nested models Is there any reason to prefer the AIC or BIC over the other?. See also AIC & BIC vs. Crossvalidation

*RESET. Use PRESS, not R squared….

*$R^2_{adj.}$. What’s a good value for R-squared?.


Tests


*

*Cross-validation techniques. 

*Lack-of-fit sum of squares. wiki.


Thanks to @Emir and @AleksandrBlekh
