Choosing number of predictor variables for regression models based on sample size and desired power

Question

Based on comments a previous question, I have changed my question.

I am wanting to see if the addition of "novel" variables to a model containing "traditional" variables improves the prediction of levels of the DV. For example, age and blood pressure are established predictors of atherosclerosis, amongst others. I include these in my base model. I then want to add kidney function and physical activity and compare the two models. I will compare the models by looking at AIC.

My question is about power, sample sizes and number of predictors. Using G power there are two options for calculating sample size in linear regression models: r squared deviation from zero, and r squared increase. I achieved a smaller sample size than planned, so now trying to calculate how many variables I can include in my models. What is the right approach here? Do I need to worry about sample size when designing models, or can I test multiple and choose the best fitting, lowest AIC, regardless of the number of variables. Which of the two options in G power is an appropriate method of seeing how many variables I can include in a model, based on desired power and sample size?

Answer 1

If I understand the situation, you have already gotten the sample. In this case, I would not do power analysis, but, rather, concern myself with overfitting.

To my mind, the main purpose of a priori sample size calculation is figuring out how big a sample you need. But your sample is determined. This then looks a lot like post hoc power analysis, which is frowned upon. But ... I'm curious what others say because, while you have the sample, you haven't done the analysis. So, this is really power analysis en media res (in the middle) and I don't know about that.