Linked Questions

0
votes
0answers
33 views

How does $\vec{\beta}=(H^TH)^{-1}H^T\vec{y}$ equivalent to least squares criteria for evaluating splines? [duplicate]

I'm learning about splines and the equation for a spline trying to predict the true function given data points is expressed as $$f(x)=\sum^k_{m=1}\beta_mh_m(x)$$ Where $\beta_m$ is some linear ...
0
votes
0answers
28 views
0
votes
0answers
10 views

OLS estimators for a linear log function [duplicate]

So I am doing some linear log functions of the form $Y_t=\beta_0 + \beta_1*log(t) + X_t$ where $X_t$ is white noise I have been using R to do the least squares estimations and that easy enough to ...
99
votes
7answers
61k views

Why use gradient descent for linear regression, when a closed-form math solution is available?

I am taking the Machine Learning courses online and learnt about Gradient Descent for calculating the optimal values in the hypothesis. h(x) = B0 + B1X why we ...
39
votes
5answers
101k views

How to derive the least square estimator for multiple linear regression?

In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
25
votes
2answers
6k views

Least Squares Regression Step-By-Step Linear Algebra Computation

As a prequel to a question about linear-mixed models in R, and to share as a reference for beginner/intermediate statistics aficionados, I decided to post as an independent "Q&A-style" the steps ...
1
vote
2answers
2k views

Full-Rank design matrix from overdetermined linear model

I'm trying to create a full-rank design matrix X for a randomized block design model starting from something like the example from page 3/8 of this paper . It's been suggested that I can go about ...
2
votes
2answers
2k views

sklearn Linear Regression vs Batch Gradient Descent

tldr: Why would sklearn LinearRegression give a different result than gradient descent? My understanding is that LinearRegression is computing the closed form solution for linear regression (...
0
votes
2answers
1k views

How is the linear regression optimize in R and Python?

I am currently working a lot with R and Python. I am not able to access the C code the the R function lm_fit. I am wondering how is the linear regression optimize in R and python ? I am pretty sure ...
2
votes
1answer
214 views

Linear algebra use case [closed]

I learning some machine learning course, and I would like to know in which case we use linea algebra and Matrix Algebra? Thank you Kind regards
0
votes
1answer
219 views

Computing overhead of statistical models for training?

Could someone provide overhead of the following model for training (With respect to input size or if there are any relevant parameters). Overhead I mean somewhat like asymptotic time complexity form. ...
1
vote
1answer
78 views

Algorithm for simple linear regression that is efficient and numerically stable

I'm developing an application that is fed with continuous data while older data is discarded. I'm using some algorithms to compute simple linear regression on these data with Perl. Basically that ...
1
vote
1answer
75 views

The best line fit can be found analytically by the least squares method. So can we say that linear regression (least squares) has an optimizer?

The best line fit can be found analytically by the least squares method. So can we say that linear regression (least squares) has an optimizer? For example, for logistic regression I can use an ...
0
votes
1answer
48 views

Computing β in multiple regression (the coefficients)

In my book I have here that $\hat\beta=(X'X)^{-1}X'Y$, and that's fine and dandy, but I have a maybe dumb question regarding this. So these $β$s are the coefficients that we must obtain from our ...