Skip to main content
Cross Validated

Return to Answer

replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
Source Link
Community Bot
Community Bot
  • 1

With linear regression, BFGS and LBFGS would be a major step backwards. That's because the solution can be directly written as

$\hat \beta = (X^T X)^{-1} X^T Y$

It's worth noting that directly using the above equation to calculate $\hat \beta$ (i.e. inverting $X^T X$ and then multiplying by $X^T Y$) is itself even a poor way to calculate $\hat \beta$.

Also, gradient descent is only recommended for linear regression in extremelyextremely special cases, so I wouldn't say gradient descent is "related" to linear regression.

With linear regression, BFGS and LBFGS would be a major step backwards. That's because the solution can be directly written as

$\hat \beta = (X^T X)^{-1} X^T Y$

It's worth noting that directly using the above equation to calculate $\hat \beta$ (i.e. inverting $X^T X$ and then multiplying by $X^T Y$) is itself even a poor way to calculate $\hat \beta$.

Also, gradient descent is only recommended for linear regression in extremely special cases, so I wouldn't say gradient descent is "related" to linear regression.

With linear regression, BFGS and LBFGS would be a major step backwards. That's because the solution can be directly written as

$\hat \beta = (X^T X)^{-1} X^T Y$

It's worth noting that directly using the above equation to calculate $\hat \beta$ (i.e. inverting $X^T X$ and then multiplying by $X^T Y$) is itself even a poor way to calculate $\hat \beta$.

Also, gradient descent is only recommended for linear regression in extremely special cases, so I wouldn't say gradient descent is "related" to linear regression.

Source Link
Cliff AB
Cliff AB
  • 21.6k
  • 1
  • 51
  • 104

With linear regression, BFGS and LBFGS would be a major step backwards. That's because the solution can be directly written as

$\hat \beta = (X^T X)^{-1} X^T Y$

It's worth noting that directly using the above equation to calculate $\hat \beta$ (i.e. inverting $X^T X$ and then multiplying by $X^T Y$) is itself even a poor way to calculate $\hat \beta$.

Also, gradient descent is only recommended for linear regression in extremely special cases, so I wouldn't say gradient descent is "related" to linear regression.