My question is what is the effect of assigning weights to observations when training a Classifier such as a Logistic Regression model.
The glm function documentation in R for example states:
Non-null weights can be used to indicate that different observations have different dispersions (with the values in weights being inversely proportional to the dispersions).
What is the meaning of this phrase?
I have read elsewhere that when an algorithm is passed with observation weights it simply treats each observation as repeated w times (w being the weight of the observation).
In a paper regarding Subsampling the authors suggest to combine Case-Control subsampling with weighting to remove the bias of the Estimator that is trained with the Case-Control subsampled data.
A simple alternative to standard case-control sampling is to weight the subsampled data points by the inverse of their probability of being sampled.
However, if Case Control subsampling is done to alleviate imbalance in the dataset (e.g. P(Y=1) = 1%) and weighting is restoring the imbalance by essentially repeating each subsampled observation what is the meaning?
My questions are not unrelated, they both relate to the meaning and effect of weighting in the context of Classification and Subsampling in particular.
Your advice will be appreciated.