How to find "theoretically best" model? Given the common problem of predicting response variable $Y$ from predictor variables $X$ and $Z$, is there any way to determine the "theoretical best" prediction possible for a response variable?  
When I am asked to find a model to do such a prediction, I might try different techniques: e.g. linear regression, KNN, etc.  However, if $X$ and $Z$ are simply not predictive at all of $Y$, then no matter how good of a model I have, it is a waste of time.    For example, if I am trying to predict a student's grade in a class, then using the temperature in Hawaii and the GDP of France will be a complete waste of time.  How can I determine that without trying it (or knowing a priori)? 
In other words, how do I find out if I should even be using $X$ and $Z$ to predict $Y$ in the first place?  Is there some way to calculate an upper bound for the "best" a model I can possibly hope to generate? 
 A: On the one hand, we will typically not know the true data-generating process, unless we have simulated the data ourselves.
Even in your example of predicting a student's grade, the temperature of Hawaii and the GDP of France may have an impact: if the weather was not nice during the student's holiday in Hawaii, he may have studied more and gotten a better grade. Or better weather may have made for a more relaxing holiday. A higher GDP in France may contribute to him doing an internship there, which could again take time away from his studies - or motivate him to do really well.
I am a firm believer in "tapering effect sizes": everything could conceivably have an impact on everything else - but the effects get weaker and weaker.
And even when we do know that a particular predictor $X$ has an impact on $Y$, sampling inaccuracy and the bias-variance tradeoff may imply that including $X$ in the model may be counterproductive. A misspecified wrong model may yield better predictions than a correct one. This is part of the reason why shrinkage works.
Bottom line: we will usually not know which variables to include for best predictive performance, and even when we know whether a variable has an impact, including it may be counterproductive. This is why modeling will stay at least partly an art.
