2
$\begingroup$

I am planning to conduct a path analysis to figure out the effect from X to Y. (not causality)

I personally believe that most statistical models should not be overfitted. Whether developing a predictive or explanatory model, overfitting should be avoided. Otherwise, the estimated parameters are not trustworthy.

However, some research paper or my laboratory member do not pay attention to this. He says that he does not consider overfitting when he fits a linear model for interpretation.

Is my idea wrong? I hope someone gives me a comment or resource for these.

$\endgroup$
1
$\begingroup$

Overfitting is predominantly an issue when building predictive models in which the goal is application to data not used to build the model itself. By definition, when the model overfits to the data that was used to create it, the generalization error of the model goes up.

As far as other applications that are not predictive, overfitting is more secondary, (although if there is no significance attached to included independent variables, you may want to look at transforming certain variables or including additional variables). It's nice when the data and the model aligns- if you have the right experiment design, you can make the argument that the variables you have included are related directly to the variation explained in your model. But far more important are things like interpretability and the fulfillment of the basic assumptions your model may make. It's why some econometricians can suffer a relatively low R^2, particularly if their model is able to show a meaningful connection between independent and dependent variables.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ thank you for your nice explanation. I have got what I wanted to know completely. I just wanna ask one more question. How should I interpret the model when the model has low R^2 and shows the meaningful connection between independent and dependent variables? Can I state that the model is not strong enough to explain dependent variables with independent one, but "X1" independent variable affects the dependent variable? $\endgroup$ – Koji Sugano Apr 16 at 4:05
  • $\begingroup$ You will have to address why R^2 is low. Possible reasons might be omitted variables or a nonlinear relationship. You'd then would want to be able to argue why you are sticking with the current model, e.g. interpretability, significant value for F-score, measure of fit, or similar outside research. I would avoid using the word "affect" unless you want to make a causal claim. And to clarify the R^2 issue, I might show a distribution of the errors. $\endgroup$ – atirvine88 Apr 16 at 11:58
  • 1
    $\begingroup$ Overfitting is still an issue, as it might lead to higher standar errors of parameter estimates, and that way hurts interpretability. $\endgroup$ – kjetil b halvorsen Apr 25 at 19:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.