I have frequently heard descriptions of the IIA problem for logit regression phrased in terms of the famous red bus/blue bus, or some direct metaphor for that thing. However, I was curious as to what this would mean practically and numerically for an actual model. Therefore I devised the following problem: Mode of transport is based on income, the wealthy drive a car and the poor ride a bus. However, the blue bus is preferred because, as everyone knows, blue is objectively superior to red. Some python code proceeds:
import numpy as np import matplotlib.pyplot as plt import sklearn import sklearn.preprocessing import sklearn.linear_model car_income = np.random.uniform(50000,100000,100) bus_income = np.random.uniform(10000,40000,100) car = np.ones(100)*0 # 0, 1 and 2 are the categorical encodings red_bus = np.ones(25)*1 blue_bus = np.ones(75)*2 y = np.hstack((car,red_bus,blue_bus)) bus = np.ones(100)*1 yy = np.hstack((car,bus)) #for graphing income = np.hstack((car_income, red_bus_income, blue_bus_income)).reshape((-1,1)) model = sklearn.linear_model.LogisticRegression(C=100000,tol=0.00000001) model.fit(income,y) plt.scatter(income,yy) X = np.linspace(-200000,200000,10000).reshape((-1,1)) plt.plot(X,model.predict_proba(X)[:,2]) # blue bus probabilities plt.show()
This models the proposed situation. The resulting plot is:
The scattered points at 0 are cars, at 1 are buses. The line is predicted probabilities for blue bus ridership at the various income levels.
The violation of the IIA assumption is the bit where blue bus is predicted at $75\%$, but then drops to $50\%$ as income goes to $-\infty$. Since the choice between red bus and blue bus is assumed to be irrelevant, the model is more or less incapable of representing the $75\%$/$25\%$ split between red bus and blue bus. However, it seems that it is almost able to do so. I would have imagined a full-on leveling off at $50\%$. How is it that the hump appears at $75\%$?