In a linear regression, how can I test whether a given variable is driving the regression's predictive power? Let's say I have at linear regression of income per capita (PPP) and other variables on food-prices
I suspect that the variable that is driving most, if not all, the prediction power of the regression is the "income per capita (PPP)" variable rather than any of the other variable. 
How can I test this hypothesis?
 A: Some day I will be able to give a more authoritative and theoretical answer to this question, but I think I can at least give food for thought. This answer is in the context of regression analysis, not machine learning or prediction per se.
If adding a variable significantly reduces the mean square error, then it is greatly enhancing the quality of your fit, and hence its likely effectiveness for prediction. You are trying to reduce the prediction interval, which is proportional to the MSE. However, there are other measures of model fit you can consider, such as adjusted $R^2$, Mallows' Cp, AIC, etc.
Depending on your goal, you may also be concerned with the effect size, which, in the case of OLS regression, is (effectively) the slope parameter. A large slope for a particular IV means that a change in the IV creates a large change in the DV. However, this scales with the units of the IV, so you need to take that into account.
More:
I didn't mention how you would do this in practice. I'm sure there are more sophisticated methods. But the simplest approach would be to take what you think is your "best" model (in terms of the IVs you are going to include, and then compare this model with leaving out one variable at a time. In R, you would do something like:
mod_all = lm(Y ~ X1 + X2 + X3 + ..., data)
mod_no_x2 = lm(Y ~ X1 + X3 + ..., data)
anova(mod_all)
anova(mod_no_x2)
anova(mod_all, mod_no_x2)

In the anova tests, you can see the mean squared error in each model. In the comparison anova (partial F-test) you can see whether removing X2 had a significant effect on the model fit. Also, summary would show you the Adjusted $R^2$. The car package has functions for AIC, etc.
The variable that has the biggest effect on the quality measure you choose is, in some sense, the one that most strongly impacts your model quality.
