This is a somewhat broad question, but I'm having trouble finding a good answer anywhere. I know many ML models will impose an independence assumption in the data. But I'm having a hard time really understanding what the practical implications are if that independence assumption is violated. For linear/logistic regression, I get that it likely biases interpretation of the coefficients. But what about from a predictive performance standpoint? Does violating the independence assumption actually matter?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
In general, violation of independence assumptions results in residuals that are not actually maximal-entropy. This could matter in terms of the applicability of known results regarding convergence to the "true" distribution being estimated.
-
$\begingroup$ Not sure if I totally follow. So it could hurt predictive power? $\endgroup$– VincentMay 14 at 14:27
-
$\begingroup$ Yes. Using a ML algorithm that assumes data are independent will fail to capture all of the details in a data set in which that assumption is violated. $\endgroup$ May 14 at 15:50