I have a dataset with a sample size of n=30, one dependent variable and 31 possible predictors.

Now I want to build a regression model as part of a regression kriging model to predict my dependent variable. Since my sample size is this small i was thinking of doing 10 fold cross validation.

Can I perform 10 fold cross validation to build the model (or in other words select the significant covariates based on the average accuracy and goodness of fit) and evaluate the prediction performance with the same data?

I don't think it is possible, also see these questions https://stats.stackexchange.com/a/179747/279199, Can cross validation be used for causal inference? but what are my options here? I have to build a prediction model and be able to evaluate the predictions.

Should I make a clear difference between causal inference and prediction? For example build different models solely based on literature and experience and then apply cross validation to see which ones perform the best? And then build a different model to detect the significant covariates and the relationships between the dependent variable and the covariates to see which covariates match with the prediction model based on literature?

  • 1
    $\begingroup$ In general, you can't make causal inference from regression. For example, you will get similar significance whether you predict y from x, or x from y, regardless of whether x causes y or y causes x. $\endgroup$
    – zbicyclist
    Apr 1, 2020 at 2:58
  • $\begingroup$ You can estimate causal effects with regression if you know your outcome follows the predictor in time and you have measured a sufficient set of confounding variables. Regression is one strategy to remove confounding. $\endgroup$
    – Noah
    Apr 1, 2020 at 5:45


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.