3
$\begingroup$

In market research I'm building a logistic regression model to estimate the likelihood that clients may change banks. The proportion of events is roughly 10% in my sample. From university I remember that a proportion of events that is too small introduces bias into the estimate. Or is it the standard error that gets biased? (Question 1)

As a rule of thumb, what is an appropriate proportion of events? (Question 2) Please give a straight and rough answer, rather than a technical explanation supported by various references.

$\endgroup$
1

1 Answer 1

2
$\begingroup$

The proportion of events is not the issue. It's the overall sample size and the number of explanatory variables. A smaller proportion requires a larger sample size. And more parameters (explanatory variables) means that you need a bigger sample size. When comparing two proportions, you would want each sample to be such that np > 10, for each sample (as a rule of thumb). This lets you have confidence in the normal approximation that you will be using for the test.

Logistic regression estimates binomial probabilities, so it is like the simple case of estimating a proportion - only more so, because you have more parameters and messier test statistics.

A general rule of thumb is 30 observations per parameter when estimating a complex model. I would up that to 100 observations per parameter in your case, using the np>10 rule. Note, that that would be a bare minimum.

RE: bias. Your software is going to do a maximum likelihood estimate of the parameters, which will work regardless of sample size. The estimates will be biased, but plausible. The problem will be the significance tests. They are based on a chi-squared approximation to the likelihood (-2 Log Lik, but you don't want technicalities), and convergence is slow with the binomial. That means that your estimates may be trustworthy but your ability to assess their significance will not, until you have a healthy sample size. (Technical point: it's not the bias of your estimates that will be the problem but the significance tests).

But my guess is that you should have plenty of data if you are using customer records from a bank. How hard would it be to get a few thousand records?

If you have access to a lot of data, you can use part of it to fit the model and part of it to test the model. If you like what you see, you don't need to worry about the convergence of the test statistics - since you won't be using them to evaluate the fit of your model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.