My problem is very simple and I have searched the web to my best capabilities but still haven't found any solutions to my problem, and sadly I have no one to ask in real life. Anyway, my problem is stated as follows: I have no idea if my logistic regression is reasonable/ correct.
I analyze horse racing as a hobby and have read about logistic regression in pretty much every paper I have read, so I thought I'd give it a shot. I get the basics of it: A linear regression is not suitable for classification problems since a linear regression will take values above 1 and below 0, this is not okay in the world of probabilities! Hence, logistic regression fixes this issue.
I've scatter plotted my variables and fitted a line to it, which is on the form y = B0+X1*B1, where X1 is my independent variable (in this case the sortPriority/ postPosition of the horse). I know that the formula for linear regression is on the form p = 1/(1+e^(-y)). The scatter looks like this: 
Blue is linear regression, orange "is" logistic regression and the green marks are my data points. The code I used is as follows:
m, b = np.polyfit(sortedDataframe[stuffToEstimate],Binary_runnerResult,1) plt.scatter(sortedDataframe[stuffToEstimate],Binary_runnerResult,marker='+', color = 'green')
Where m is the intercept and b is the slope of the fitted line.
plt.plot(np.linspace(0,500,1000),[b+m*i for i in np.linspace(0,500,1000)])
plt.plot(np.linspace(0,500,1000),[1/(1+np.e**(-(b+m*i))) for i in np.linspace(0,500,1000)])
Now, this might be a stupid question, but does this even look somewhat reasonable/ correct? In all the tutorials I've watched they all get this really nice S-shape, which makes sense because their datasets "looks cleaner" than mine. This might be because the nature of horse racing is very different from calculating if you'll get a mortgage based on credit score, i.e. people with low credit score almost never get a mortgage and people with high credit score almost always get a mortgage whereas a horse with low sortPriority doesn't almost always win a race. Though I feel that I should have a stronger S-shape then I have since sortPriorities 5-15 never won a race. I also have this plot where the issue might not be as obvious: 
So back to my question: How would I make this better?
I realize that I cant do a logistic regression on all my data all at once, the regressions in the plot for example are from races on 2000m distance and on a specific class. But I'm afraid that I will overfit my model if I add more restrictions on the dataset.
Last question: Is it even correct to pick a dataset that contains all of the horses of a race or should I maybe pick individual horses that fit what I want to regress?
Sorry for the wall of text but any help would be much appreciated. Cheers!
