Dealing with outlier data causing a non-linearity for logistic regression model

Question

I'm building a logistic regression model where yes is the target, one of the attributes spnr_avg_spend_mod is shown below. It is a continuous variable but has been binned up into 10 bins using pd.qcut.

As shown in the image below, apart from the first data point there is a linear decrease in the proportion of yes with increasing spend (increasing spnr_avg_spend_mod).

I am looking for ideas/methods on how to deal to this when modelling as it is a nice feature apart from that point.

In [14]: df 
Out[25]: 
   spnr_avg_spend_mod       yes
0                   0  0.474293
1                   1  0.531138
2                   2  0.533260
3                   3  0.503260
4                   4  0.503418
5                   5  0.482936
6                   6  0.479729
7                   7  0.460062
8                   8  0.450755
9                   9  0.421202

In [15]: plt.scatter(df.spnr_avg_spend_mod, df.yes)
Out[15]: <matplotlib.collections.PathCollection at 0x7f80962873c8>

Answer 1

Binning a continuous IV is almost always a mistake. See Frank Harrell's book Regression Modeling Strategies where he lists 11 problems with this and sums up "Nothing could be more disastrous". Leave the IV continuous and then you can try using a spline of it as a predictor.

Answer 2

Since there isn't a textbook solution to what you're asking, I'll offer a few considerations:

1. How do you know that data point is an outlier? Do you know for certain that your trend must be linear? There are numerous phenomena that are non-linear. Your logistic regression will pick that up if it's true.
2. Have you tried using a different number of bins? Do you know for a fact that this feature should be cut into deciles? For example, if you had 3 or 4 bins, there's a good chance that the resulting bins would be linear. Similarly, if you had 20 bins, then perhaps only the first bin or two would be lower than expected, which helps identify the cause of a potential outlier.
3. Related to 2., in my experience data on the edges (e.g., first and last bin) can behave differently than the middle because of their proximity to the borders. That is, the first bin may include 0's or negatives which may have unexpected results on your output variable. Have you done the necessary data cleaning required?

Answer 3

This is my comparison of a third order polynomial and a different equation from an equation search, placed here as I cannot display images in the comments. The third order polynomial does not visually appear to fit the shape of the data as well, especially at the higher end.

First the third order polynomial:

import math
def Polynomial_Cubic_model(x_in): # from zunzun.com
    temp = 0.0

    # coefficients
    a = 4.8789645034964935E-01
    b = 3.1491002913752966E-02
    c = -8.4664254079254170E-03
    d = 4.7255536130533238E-04

    temp += a + b * x_in + c * math.pow(x_in, 2.0) + d * math.pow(x_in, 3.0)
    return temp

and here the equation search result:

import math
def Simple_SimpleEquation_18_model(x_in): # from zunzun.com
    temp = 0.0

    # coefficients
    a = 4.7730833313413012E-01
    b = -6.6062232805066859E-02
    c = 1.5999550517794348E-01

    temp = a*math.exp(b*x_in+c*math.pow(x_in,0.5))
    return temp