Is a variable significant in a linear regression model? I've got a linear regression model with the sample and variable observations and I want to know:


*

*Whether a specific variable is significant enough to remain included in the model.

*Whether another variable (with observations) ought to be included in the model.


Which statistics can help me out? How can get them most efficiently?
 A: I second Rob's comment. An increasingly prefered alternative is to include all your variables and shrink them towards 0. See Tibshirani, R. (1996). Regression shrinkage and selection via the lasso.
http://www-stat.stanford.edu/~tibs/lasso/lasso.pdf
A: For part 1, you're looking for the F-test. Calculate your residual sum of squares from each model fit and calculate an F-statistic, which you can use to find p-values from either an F-distribution or some other null distribution that you generate yourself.
A: Statistical significance is not usually a good basis for determining whether a variable should be included in a model. Statistical tests were designed to test hypotheses, not select variables. I know a lot of textbooks discuss variable selection using statistical tests, but this is generally a bad approach. See Harrell's book Regression Modelling Strategies for some of the reasons why. These days, variable selection based on the AIC (or something similar) is usually preferred.
A: Another vote for Rob's answer.
There are also some interesting ideas in the "relative importance" literature.  This work develops methods that seek to determine how much importance is associated with each of a number of candidate predictors.  There are Bayesian and Frequentist methods.  Check the "relaimpo" package in R for citations and code.
A: I also like Rob's answer.  And, if you happen to use SAS rather than R, you can use PROC GLMSELECT for models that would be done with PROC GLM, although it works well for some other models, as well.  See
Flom and Cassell "Stopping Stepwise: Why Stepwise Selection Methods are Bad and What you Should Use" presented at various groups, most recently, NESUG 2009
