Is correlation the best way to determine if variables will not add new information to a model? When feeding two variables (features) to a model (could be linear regression, GBM, Random Forest), is (Pearson) correlation between the two variables the best way to determine that the second variable will not add new information to the model?
 A: This question is related to the "feature selection", also known as "model selection" in regression problems. First, we test the significance of individual coefficients $\beta_{i}$ that is correspondent to the feature $x_i$. This is a test of hypothesis, where the null and alternative hypotheses are as follows: 
$H_0 : \beta_i = 0$ 
$H_1 : \beta_i \neq 0$
This is a t-test, for which the t-statistic is computed as follows: 
$t_i = \frac{\hat{\beta}_i}{\sigma(\hat{\beta}_{i})},$
where $\hat{\beta}_i$ is your linear regression estimate and $\sigma(\hat{\beta}_{i})$ is the standard error related to that estimate. 
At that stage, if multiple features ended up being "significant", a "multicollinearity" test is often handy to detect whether the features are significantly correlated with each other. 
A common measure of collinearity is the variance inflation factor, which is a function of the coefficient of determination $R^2$. 
For more information about how to compute VIF's, take a look at the wikipage: 
Variance inflation factor 
You can also run some diagnostics on what people call "the hat matrix", which is $X^TX$, where $X$ is your $N*p$ design matrix such that $N$ is the number of observations and $p$ is the number of features. The inverse of that matrix ends up to be the correlation matrix of the regression coefficients. If a non-diagonal entry of that correlation matrix is significantly non-zero, then there is a good chance that two of your features are strongly correlated. 
Another similar technique is to do a sequential model adequancy test, by "sequentially" adding variables one at a time and checking how much benefit, in terms of reduction in uncertainty, you get by adding one variable, given the other variables. This is an F-test, which is the essence of "forward and stepwise" model selection techniques. 
