I am quite inexperienced using logistic regression and am having trouble understanding my data and how the regression behaves. Here's the outline of my problem:

I have a (medical) test that gives a yes/no result (basically like a pregnacy test) and I have a continuous variable that I want to use to predict the test result. Furthermore, I want to divide my data into two subsets (according to another attribute) and check whether the test gives different results based on subset membership. I ran a logistic regression on these two subsets respectively (using python's statsmodels package) to compare the output. Looking at the plots, the fitted lines for each subset are quite a bit apart (sorry this is not very specific).

The subsets are quite different from each other wrt the test results, i.e. subset 1 has six times more positive than negative results whereas for subset two it's only about 1.5 times more. I then manipulated the data of the two subsets in such a way that I have equal numbers of positive and negative test results (by excluding positive datapoints randomly) in each of them. When I now run the two regressions, the lines are basically the same.

I am wondering what this tells me or if I can draw any conclusions from it, i.e. "the test works the same given the continuous independent variable, irrespective of subset membership". But this is only true when I balance the datasets and the output of the regression is clearly different with the original data. Basically, I am wondering if this difference is due to different distributions of the continuous variable in the two subsets (which are different, i.e. they have different shapes but means and medians are relatively close to each other) or if the test actually does not only work based on what the continuous variable represents but is affected by something else as well (which I think is really unlikely).

I hope I'm somewhat making sense. Maybe someone has an alternative suggestion of what to do to investigate this question and some insight into what could explain such a behaviour or where I'm making a mistake in my thought process.

Thanks in advance!


1 Answer 1


A few things:

  • Do not split your data. You should be using the attribute you're using to split the data as a variable in the model.

  • Do not manipulate the outcome. Analyze the data as they are.

  • I would not attempt to interpret the results of your procedure as the frequency properties of the procedure are not well understood. If you can post your data, or a reasonable facsimile, we can show you an appropriate method of analyzes.

  • $\begingroup$ Thanks a lot for your reply! I've already included subset membership as an additional attribute (let's call it "att") and the p-value for the corresponding coefficient was 0.002. However, I think "att" affects the value of the continuous variable ("cv"), I don't know if this is a problem. The thing is that the test is designed to work on "cv", so att having an effect (given "cv") doesn't really make any sense to me. Anyway, I don't feel comfortable posting any sort of data here and I understand that it's hard to help without that. But your answer was already very helpful, thanks again! $\endgroup$
    – MyLion
    Commented Mar 20, 2021 at 20:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.