Different versions of forward stepwise regression

Question

Forward stepwise regression is a popular method but I found at least three different versions. I was wondering which one is the most popular, and which one is implemented in R.

1. Let $p$ be the total number of covariates. One version states that, given $k$ covariates have been selected, find the one from $p-k$ covariates that are most correlated with the residual. Then include this variable and regress the old residual on this new variable, yielding a new residual. Meanwhile keep all the original $k$ fitted coefficients the same.

2. Another version differs from the first version in that, after the new variable is included, fit the response $\hat{y}$ on the $k+1$ variables. In other words, the $k$ previously included covariates have coefficients updated too.

3. I also encountered a version as follows. For each of the $p-k$ covariates, consider the augmented model formed by adding this covariate to the $k$ original covariates. Then regress the response $y$ on the {\em $k+1$} covariates. Among these $p-k$ least-squares fits, select the covariate which yields the best fit. Note that this is quite computationally intensive.

Answer 1

Following up on @ttnphns: it sounds like in #1 & #3, the coefficients on the $k$ original predictors are not updated, correct? This is not a version of stepwise regression I have ever seen. Every time I see stepwise regression, all coefficients are updated.

And then, as @ttnphns notes, there is no difference between choosing the new covariate that is most strongly correlated with the residuals and choosing the one that yields the largest increase in $R^2$.

(You do know that inferential statistics and hypothesis-testing are invalid after stepwise-regression unless you correct in non-trivial ways for the stepwise variable selection, right?)