I have only heard about dropout being applied to training of neural networks. Could the same technique, in theory, be applied to any iterative ML algorithm? For example, in mini-batch training, each mini-batch iteration could randomly drop out some arbitrary features. Has this been tried for, say, logistic regression with SGD optimization? Any thoughts/opinions appreciated.
Wager et al. seem to follow the approach I suggested in the OP: