0
$\begingroup$

This is a somewhat broad question, but I'm having trouble finding a good answer anywhere. I know many ML models will impose an independence assumption in the data. But I'm having a hard time really understanding what the practical implications are if that independence assumption is violated. For linear/logistic regression, I get that it likely biases interpretation of the coefficients. But what about from a predictive performance standpoint? Does violating the independence assumption actually matter?

$\endgroup$

1 Answer 1

1
$\begingroup$

In general, violation of independence assumptions results in residuals that are not actually maximal-entropy. This could matter in terms of the applicability of known results regarding convergence to the "true" distribution being estimated.

$\endgroup$
2
  • $\begingroup$ Not sure if I totally follow. So it could hurt predictive power? $\endgroup$
    – Vincent
    May 14 at 14:27
  • $\begingroup$ Yes. Using a ML algorithm that assumes data are independent will fail to capture all of the details in a data set in which that assumption is violated. $\endgroup$ May 14 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.