2
$\begingroup$

Just wonder if you could recommend a few methods (other than tree-based methods) to analyze a dataset in which n= 350 and p = 35. The goal is not so much about prediction, but to find/select predictors that are strongly associated with the response. The response variable is work-related injury rates of an occupation, a positive continuous number. There are about 35 predictors, each of which characterizes some feature of a given occupation, such as the degree to which physical strength is required at the job; some of these predictors are highly correlated. There are about 350 occupations in the dataset. The response and predictors are extracted from two different databases and merged together by occupation. Both the predictors and response are aggregated data, which usually contain a higher level of errors than individual-level data.

I have tried to use regression trees to select the variables (see the link please). But the level of cross-validation errors was exceedingly high, indicating the dataset may had a high level of irreducible errors. Consequently, tree-based methods, while flexible, may be fitting those errors, leading to poor cross-validation results. I’ve also tried random forest and the result was still not satisfactory. It’s not clear to me where the irreducible errors come from. One possibility is that aggregating over observations (for the predictors and response) creates additional errors. Further, there could be many factors that contribute to injury risk at work which are not reflected by the 35 predictors (e.g., health status of workers in general in the occupation), which provides another sources of errors.

If the level of irreducible errors is high, then maybe a method less flexible than tree-based methods would work better. I am considering using the following: (1) step-wise regression, (2) Lasso, or (3) ridge regression. Any other methods you would recommend? I’d appreciate it. Thanks.

$\endgroup$
4
  • $\begingroup$ Point of interest: lasso and ridge regression are both special cases of elastic net regression, but these methods will probably be worse at selection than tree-based methods, since the functional form of the predictor must be in the form of a multilinear system of equations. This thread is also related: stats.stackexchange.com/questions/164048/… $\endgroup$
    – Sycorax
    Nov 16, 2015 at 15:53
  • $\begingroup$ I really like the Boruta algorithm (available in R). $\endgroup$
    – neuron
    Nov 16, 2015 at 16:32
  • 3
    $\begingroup$ With the sample size you have it is a mistake to assume that any variable selection method will select the "right" variables. Repeated re-analysis (from scratch) using the bootstrap will show extreme instability in the list of variables "selected". $\endgroup$ Nov 20, 2015 at 14:36
  • $\begingroup$ Thanks so much, Dr. Harrell. I can double or triple the number of the y's for each given occupation by combining several rounds of the response data. So for a given occupation, it will have an injury rate in 2010, and another in 2011, or 2012, ..etc. Would this make the sample size good enough for the analysis? The predictors have only one round data, though; so it will be like using the predictors in 2010 to predict the responses which are combined using 2010-2012 data. $\endgroup$
    – TCL
    Nov 20, 2015 at 16:10

1 Answer 1

1
$\begingroup$

Piironen and Aki recently published a paper comparing several variable selection methods from a Bayesian perspective. They ultimately recommend fitting a full model and projecting its information onto sub-models; further details and a working example with R and Stan code are provided in the supplement to their paper. A subsequent publication details an extension of the method to Gaussian Process models.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.