Adding weights to logistic regression for imbalanced data I want to model a logistic regression with imbalanced data (9:1).
I wanted to try the weights option in the glm function in R, but I'm not 100% sure what it does.
Lets say my output variable is c(0,0,0,0,0,0,0,0,0,1).
now I want to give the "1" 10 times more weight.
so I give the weights argument weights=c(1,1,1,1,1,1,1,1,1,1,1,10).
When I do that, it will be considered in the calculation of the maximum likelihood. Am I right? misclassification of "1" is just 10 times worse then missclassifying a "0".
 A: Weighting is a procedure that weights the data to compensate for differences in sample and population (King 2001). For example, in rare events (such as fraud in credit risk, deaths in medical literature) we tend to sample all the 1’s (rare events) and a fraction of 0’s (non events). In such cases we have to weight the observations accordingly.
Example:  Let us say, In a population of 500,000 transactions there are 50 fraud transactions. In this case you would    


*

*Sample all 50 frauds transaction (100% of the fraud)

*10% of the good transactions (10% of 500,000 is 50,000 good transactions)


In this case you would assign a weight of 1 for fraud transactions and a weight of 10 for good transactions. This is called the Weighted Maximum Likelihood method. The important takeaway is that the weighting is related to sampling proportions
Refer: Logistic Regression in Rare Events Data (King 2001)
A: Ching, You do not have to make your data set balanced in terms of  1’s and 0’s. All you need is sufficient number of 1’s for the maximum likelihood to converge.  Looking at the distribution of 1’s (100,000) in your dataset, you should not have any problems. You can do a simple experiment here


*

*Sample 10 % of the 1’s and 10% of the 0’s and use a weight of 10 for both

*Sample 100% of the 1’s and 10% of the 0’s and use a weight of 10 for the 0’s


In both cases, you will get identical estimates. Again the idea of weighting is related to sampling. If you are using the whole data set you should not weight it. If I were you I would just use 10% if 1's and 10% of 0's.
In R, you would use glm. Here is a sample code:
glm(y ~ x1 + x2, weights = wt, data =data, family = binomial("logit"))

In your dataset there should be a variable wt for weights.
If you use 10% of both 0's and 1's, your wt variable will have a value of 10.
If you use 10% of the 0's and 100% of 1's: wt variable will have a value of 10 for observations with y=0 and 1 for observations with y=1
