Initial guess for linear regression This is a very basic question but I was reading the notes by Andrew NG (https://see.stanford.edu/materials/aimlcs229/cs229-notes1.pdf) about linear regression and am wondering how the initial guess for theta is made by algorithms such LinearRegression in python sklearn library or in general if I try to manually calculate this.
I understand using gradient descent will eventually lead to almost the same result for theta regardless of the initial guess but what I want to know is how these algorithms make the initial guess, be it for the intercept or the coefficient of one of the parameters.
(Here, theta refers to the coefficient for various predictor variables or the intercept)
 A: sklearn's vanilla LinearRegression uses LAPACK, and so (if I understand correctly) does not use gradient descent, but a closed-form solution. However, their penalized versions, as well as logistic regression (penalized or not) do use either gradient descent or coordinate descent.
The initialization of coefficients might depend on the solver, but the two places I see sklearn explicitly initializing coefficients, it sets them to zero:
if not self.warm_start or not hasattr(self, "coef_"):
    coef_ = np.zeros((n_targets, n_features), dtype=X.dtype,
                     order='F')

source for the above, in ElasticNet.fit, and similar source in enet_path,
the coordinate descent elastic-net path function.
Their logistic regression models use external solvers like liblinear, saga, etc., and I'm not sure how those are initialized.  the LIBLINEAR paper only mentions initialization as "Given initial $w\in R^n$."
Setting your own initial guess can be accomplished when the sklearn estimator has a warm_start option, as you might guess from the above code snippet: just manually set the attribute coef_ to your initial guess.
