0
$\begingroup$

I'm running a multiple logistic regression with a binary output 0,1 with inputs of both continuous and discrete variables- around 6 independent variables total. The output is if a customer will buy or not, but there are a lot of results in which they bought at one point (1) and then decided to no longer buy (0). As such there are repeated values of the independent variables for both choices. Further, this portion of the dataset is large- 850 repeats/8000 overall data points. I am able to get significant results with the data included, but was wondering what the best practice is. Mainly, should this data be removed, should only one of the data points be used (perhaps 1 since they bought at one point), and what is the effect on the model in regards to adding noise or statistical issues? Thank you so much!

Edit

Thank you for the reply, I realize my initial question was very unclear. I am taking goodness of fit into account- though I do not fully understand how removing data from this effects the accuracy of the model or could skew the model. For example I'll compare the original full dataset (6508 data points), a subset of data keeping the duplicated values only when they were a customer (i.e. removing them from no longer being a customer*)(5498 data points), and a subset of the data with all duplicates removed (4655 data points). *The reasoning for the middle option is that they chose to be a customer at one point, and the reason for them no longer continuing to be a customer can be complex/external to why they initially chose to be a customer. The Hosmer-Lemeshow Goodness-of-Fits are below (holding variable selection constant/model):

Full Data

Hosmer-Lemeshow Goodness-of-Fit Test

Chi-squared: 5.281 df: 8
p-value: 0.727

Summary: model seems to fit well.

Keeping Duplicates as 1's

Hosmer-Lemeshow Goodness-of-Fit Test

Chi-squared: 10.295 df: 8
p-value: 0.245

Summary: model seems to fit well.

Removing all duplicates

Hosmer-Lemeshow Goodness-of-Fit Test

Chi-squared: 10.915 df: 8
p-value: 0.207

Summary: model seems to fit well.

From this we see that the best P-value seems to be from keeping all of the data in, but I was curious if this is bad practice or needs further considerations. @Frank-Harrell brilliantly pointed out such duplications may need to be considered by modeling with a random effects binary logistic model. Would this be the way to move forwards considering that most customers will not be repeated (though this would mean that new customers would have no data in regards to them and would be the random effect). The tradeoff here would be that the model already has a very low psuedo-R^2 and would introducing mixed effects make this intractable? My apologies for my lack of knowledge into mixed effects, I have not had experience with them prior. Thank you!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

That fact that you judge the performance of the method by an ability to extract "significant" results is emblematic of a deeper problem. Goodness of fit is far more important, and this needs to be judged by the way the predictors are fit as well as the way you model the correlation pattern within-customer. Note that 8000 customer-times is not huge for a low-information binary endpoint.

To make use of serial within-person data you should choose a model that matches the correlation pattern, typically a random effects binary logistic model or a Markov process, the latter handling serial correlation better, e.g., it handles the situation where the correlation between two outcomes is higher when assessed at smaller time gaps between them. Random effects models assume an equal correlation pattern, i.e., correlation is independent of time gap (compound symmetric correlation pattern).

Be sure not to assume linearity for continuous predictors. Consider the use of regression splines for example.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for this incite! I could use a customerID for this mixed effect modelling, though since most customers will be new... and hopefully long term the ratio of new customers to "repeat customers" will be an unimportant level, would this still be the way? From my limited understanding mixed effects are useful for variations in effects across groups/beyond initial data. The best short example I saw was how measured blood pressure could vary across doctorIDs but in this case most doctorIDs exist. But here most most customerIDs will be new. Apologies for my lack in understanding. $\endgroup$
    – WVtinker
    Jun 12 at 19:30
  • $\begingroup$ Random effects models can benefit from having a large number of subjects with only one measurement as long as you also have a large number of subjects with multiple measurements. As an alternative a first-order Markov process needs to almost always have a "state zero" (outcome state at baseline time) plus at least one later measurement. Also briefly consider GEE. $\endgroup$
    – Frank Harrell
    Jun 13 at 11:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.