I have a dataset of about 7,900 rows, which includes 28 independent variables. This is a naturally occurring data that I am trying to explore which of these independent variables have significant effect on the dependent variable. I.e., I do not have any specific theory in mind that explains relationships that I can test. I am trying to figure out which independent variables and interactions I should enter the regression model?
I have already read this thread and this thread, but considering the fact that my case is exploratory rather than predicting or testing a specific hypothesis, I do not think the approaches suggested in the following threads can help me.
One approach that came to my mind was to write a loop that enters different combinations of the independent variables and checks the AIC or BIC of the model. At the end it chooses the model with the lowest AIC or BIC. I think the issue with this idea might be overfitting the data.
I'll appreciate it if you help me with answering this question or giving me more references to learn about this.