Can overfitting be reduced using LDA I was attending a workshop on 'credit rating models' where the professor happened to mention the following: Logit models overfit the data as multicollinearity is disregarded. I am not entirely convinced with the statement. Can you please explain if(then why) the statement is true. 

 A: It can be instructive for a question like this to consider extreme cases. Suppose $X_1$ and $X_2$ are your two predictors and that $$logit(p)=X_1.$$
That is, $X_1$ has a coefficient of 1 and $X_2$ has a coefficient of 0. Now suppose that the predictors are highly correlated, and in your data set in particular they are perfectly correlated. In this case, any model of the form
\begin{equation}
logit(p)=\beta X_1+(1-\beta)X_2
\end{equation}
will explain the observed data equally well. You could pick one at random, but if it happens to be one where $\beta$ is small, you'll get poor predictions for future observations where $X_2$ is large and $X_1$ is small (or vice versa).
Now suppose you aren't so unlucky and your initial sample doesn't have $X_1$ and $X_2$ perfectly correlated, they're merely very strongly correlated as in the population as a whole. In this case, not all values of $\beta$ will be equally good, so the algorithm used to fit the model should find a single 'best' set of coefficients. These will, unfortunately, be very sensitive to the exact response values and small amounts of random variation can translate into dramatic differences in the coefficient estimates. This, in turn, leads to regions of the predictor space where estimation error is very high.
