Just wonder if you could recommend a few methods (other than tree-based methods) to analyze a dataset in which n= 350 and p = 35. The goal is not so much about prediction, but to find/select predictors that are strongly associated with the response. The response variable is work-related injury rates of an occupation, a positive continuous number. There are about 35 predictors, each of which characterizes some feature of a given occupation, such as the degree to which physical strength is required at the job; some of these predictors are highly correlated. There are about 350 occupations in the dataset. The response and predictors are extracted from two different databases and merged together by occupation. Both the predictors and response are aggregated data, which usually contain a higher level of errors than individual-level data.
I have tried to use regression trees to select the variables (see the link please). But the level of cross-validation errors was exceedingly high, indicating the dataset may had a high level of irreducible errors. Consequently, tree-based methods, while flexible, may be fitting those errors, leading to poor cross-validation results. I’ve also tried random forest and the result was still not satisfactory. It’s not clear to me where the irreducible errors come from. One possibility is that aggregating over observations (for the predictors and response) creates additional errors. Further, there could be many factors that contribute to injury risk at work which are not reflected by the 35 predictors (e.g., health status of workers in general in the occupation), which provides another sources of errors.
If the level of irreducible errors is high, then maybe a method less flexible than tree-based methods would work better. I am considering using the following: (1) step-wise regression, (2) Lasso, or (3) ridge regression. Any other methods you would recommend? I’d appreciate it. Thanks.
