2 replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
source | link

Although I recommended lm.ridge to you in response to an earlier question, you might consider the glmnet package as a better way to get started with ridge regression in R. It has the advantage that you can then follow along with the examples in Chapter 6 of An Introduction to Statistical Learning. The package also has functions for cross-validation of your model. Going through this process will help you understand the issues involved in choosing the penalty value for your ridge regression. It's not as easy as getting an automatic answer, but at the end you will have a better understanding of how the final model was built and be better able to explain (or defend) it to others. And glmnet also allows combinations of ridge and LASSO regression that some favor.

If your interest is in using your model for prediction, then measures of prediction error from cross-validation or bootstrapping are much more important than p-values for individual coefficients. For prediction you want to keep all predictors in the model (if there aren't too many), provided you have minimized overfitting with an approach like ridge regression, rather than omit those with nominally "insignificant" p-values. The p-values reported by linearRidge in the ridge package are based on an algorithm in a paper by the package's author that does not seem to have received much attention, and the package is presently orphaned as the author's email reported to the R repository at CRAN no longer works. So I'd be a bit hesitant to rely on those p-values or that unmaintained package.

If you do need coefficient p-values for some reason, bootstrapping would be a better approach. Once you have an algorithm for choosing the coefficients in a ridge regression, you make multiple bootstrap samples (with replacement) of the same size from your original data, then repeat the entire process to get the regression coefficients for each bootstrap sample with the boot package. You will have to write a function to report the regression coefficients, but that's reasonably straightforward. The distributions of those regression coefficients among the bootstrap analyses provides an estimate of their confidence limits, with the boot.ci function in the boot package. That way you incorporate variability arising from all steps of the modeling process into the coefficient error estimates.

But even those bootstrapped p-values could be misleading, as they would ignore the tradeoffs among coefficients of collinear predictors, which you evidently face based on your earlier questionearlier question. With even just 2 collinear predictors, their individual regression coefficients are likely to vary widely among bootstrap samples so that their individual p-values may appear insignificant. Nevertheless, their joint contributions to the regression might be much more stable and thus their combination very significant in practical terms. So think really hard about whether you really are interested in p-values for individual coefficients.

Although I recommended lm.ridge to you in response to an earlier question, you might consider the glmnet package as a better way to get started with ridge regression in R. It has the advantage that you can then follow along with the examples in Chapter 6 of An Introduction to Statistical Learning. The package also has functions for cross-validation of your model. Going through this process will help you understand the issues involved in choosing the penalty value for your ridge regression. It's not as easy as getting an automatic answer, but at the end you will have a better understanding of how the final model was built and be better able to explain (or defend) it to others. And glmnet also allows combinations of ridge and LASSO regression that some favor.

If your interest is in using your model for prediction, then measures of prediction error from cross-validation or bootstrapping are much more important than p-values for individual coefficients. For prediction you want to keep all predictors in the model (if there aren't too many), provided you have minimized overfitting with an approach like ridge regression, rather than omit those with nominally "insignificant" p-values. The p-values reported by linearRidge in the ridge package are based on an algorithm in a paper by the package's author that does not seem to have received much attention, and the package is presently orphaned as the author's email reported to the R repository at CRAN no longer works. So I'd be a bit hesitant to rely on those p-values or that unmaintained package.

If you do need coefficient p-values for some reason, bootstrapping would be a better approach. Once you have an algorithm for choosing the coefficients in a ridge regression, you make multiple bootstrap samples (with replacement) of the same size from your original data, then repeat the entire process to get the regression coefficients for each bootstrap sample with the boot package. You will have to write a function to report the regression coefficients, but that's reasonably straightforward. The distributions of those regression coefficients among the bootstrap analyses provides an estimate of their confidence limits, with the boot.ci function in the boot package. That way you incorporate variability arising from all steps of the modeling process into the coefficient error estimates.

But even those bootstrapped p-values could be misleading, as they would ignore the tradeoffs among coefficients of collinear predictors, which you evidently face based on your earlier question. With even just 2 collinear predictors, their individual regression coefficients are likely to vary widely among bootstrap samples so that their individual p-values may appear insignificant. Nevertheless, their joint contributions to the regression might be much more stable and thus their combination very significant in practical terms. So think really hard about whether you really are interested in p-values for individual coefficients.

Although I recommended lm.ridge to you in response to an earlier question, you might consider the glmnet package as a better way to get started with ridge regression in R. It has the advantage that you can then follow along with the examples in Chapter 6 of An Introduction to Statistical Learning. The package also has functions for cross-validation of your model. Going through this process will help you understand the issues involved in choosing the penalty value for your ridge regression. It's not as easy as getting an automatic answer, but at the end you will have a better understanding of how the final model was built and be better able to explain (or defend) it to others. And glmnet also allows combinations of ridge and LASSO regression that some favor.

If your interest is in using your model for prediction, then measures of prediction error from cross-validation or bootstrapping are much more important than p-values for individual coefficients. For prediction you want to keep all predictors in the model (if there aren't too many), provided you have minimized overfitting with an approach like ridge regression, rather than omit those with nominally "insignificant" p-values. The p-values reported by linearRidge in the ridge package are based on an algorithm in a paper by the package's author that does not seem to have received much attention, and the package is presently orphaned as the author's email reported to the R repository at CRAN no longer works. So I'd be a bit hesitant to rely on those p-values or that unmaintained package.

If you do need coefficient p-values for some reason, bootstrapping would be a better approach. Once you have an algorithm for choosing the coefficients in a ridge regression, you make multiple bootstrap samples (with replacement) of the same size from your original data, then repeat the entire process to get the regression coefficients for each bootstrap sample with the boot package. You will have to write a function to report the regression coefficients, but that's reasonably straightforward. The distributions of those regression coefficients among the bootstrap analyses provides an estimate of their confidence limits, with the boot.ci function in the boot package. That way you incorporate variability arising from all steps of the modeling process into the coefficient error estimates.

But even those bootstrapped p-values could be misleading, as they would ignore the tradeoffs among coefficients of collinear predictors, which you evidently face based on your earlier question. With even just 2 collinear predictors, their individual regression coefficients are likely to vary widely among bootstrap samples so that their individual p-values may appear insignificant. Nevertheless, their joint contributions to the regression might be much more stable and thus their combination very significant in practical terms. So think really hard about whether you really are interested in p-values for individual coefficients.

1
source | link

Although I recommended lm.ridge to you in response to an earlier question, you might consider the glmnet package as a better way to get started with ridge regression in R. It has the advantage that you can then follow along with the examples in Chapter 6 of An Introduction to Statistical Learning. The package also has functions for cross-validation of your model. Going through this process will help you understand the issues involved in choosing the penalty value for your ridge regression. It's not as easy as getting an automatic answer, but at the end you will have a better understanding of how the final model was built and be better able to explain (or defend) it to others. And glmnet also allows combinations of ridge and LASSO regression that some favor.

If your interest is in using your model for prediction, then measures of prediction error from cross-validation or bootstrapping are much more important than p-values for individual coefficients. For prediction you want to keep all predictors in the model (if there aren't too many), provided you have minimized overfitting with an approach like ridge regression, rather than omit those with nominally "insignificant" p-values. The p-values reported by linearRidge in the ridge package are based on an algorithm in a paper by the package's author that does not seem to have received much attention, and the package is presently orphaned as the author's email reported to the R repository at CRAN no longer works. So I'd be a bit hesitant to rely on those p-values or that unmaintained package.

If you do need coefficient p-values for some reason, bootstrapping would be a better approach. Once you have an algorithm for choosing the coefficients in a ridge regression, you make multiple bootstrap samples (with replacement) of the same size from your original data, then repeat the entire process to get the regression coefficients for each bootstrap sample with the boot package. You will have to write a function to report the regression coefficients, but that's reasonably straightforward. The distributions of those regression coefficients among the bootstrap analyses provides an estimate of their confidence limits, with the boot.ci function in the boot package. That way you incorporate variability arising from all steps of the modeling process into the coefficient error estimates.

But even those bootstrapped p-values could be misleading, as they would ignore the tradeoffs among coefficients of collinear predictors, which you evidently face based on your earlier question. With even just 2 collinear predictors, their individual regression coefficients are likely to vary widely among bootstrap samples so that their individual p-values may appear insignificant. Nevertheless, their joint contributions to the regression might be much more stable and thus their combination very significant in practical terms. So think really hard about whether you really are interested in p-values for individual coefficients.