Linked Questions
19 questions linked to/from What algorithm is used in linear regression?
1
vote
0
answers
108
views
What are the calculations or maths behind least-squares-minimizing in linear regression used by sklearn [duplicate]
I'm relatively new in the ML field, and this question came up when working with linear regression from sklearn library.
After a bit of looking up in the documentation, it states
Compute least-squares ...
- 11
0
votes
0
answers
67
views
Finding inverse matrix $(X'X)^{-1}$ with $X$ as design matrix [duplicate]
I'm relatively new to all this and I am trying to figure out how I can derive the matrix $(X'X)^{-1}$ when I have given $x_1, x_2, x_3$ and $y$. $X$ is the design matrix in that case but not sure how ...
0
votes
0
answers
35
views
How does $\vec{\beta}=(H^TH)^{-1}H^T\vec{y}$ equivalent to least squares criteria for evaluating splines? [duplicate]
I'm learning about splines and the equation for a spline trying to predict the true function given data points is expressed as
$$f(x)=\sum^k_{m=1}\beta_mh_m(x)$$
Where $\beta_m$ is some linear ...
- 730
0
votes
0
answers
29
views
46
votes
5
answers
145k
views
How to derive the least square estimator for multiple linear regression?
In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
- 859
27
votes
2
answers
7k
views
Least Squares Regression Step-By-Step Linear Algebra Computation
As a prequel to a question about linear-mixed models in R, and to share as a reference for beginner/intermediate statistics aficionados, I decided to post as an independent "Q&A-style" the steps ...
- 26.9k
6
votes
2
answers
782
views
Can duplicate examples create multi-collinearity?
We know if any linear dependency exists in training data, therefore, the feature matrix becomes singular and hence, we cannot solve it. But apart from the features (columns), a matrix can still can be ...
- 235
4
votes
1
answer
5k
views
Single layer neural network as linear regression
I'm really struggling to see the analogy between linear regression and a single layer perceptron. They are supposedly the same thing.
I completely understand the concept of the inputs to the neuron ...
- 555
1
vote
2
answers
5k
views
Full-Rank design matrix from overdetermined linear model
I'm trying to create a full-rank design matrix X for a randomized block design model starting from something like the example from page 3/8 of this paper (Wayback Machine) .
It's been suggested that I ...
- 163
2
votes
2
answers
3k
views
sklearn Linear Regression vs Batch Gradient Descent
tldr: Why would sklearn LinearRegression give a different result than gradient descent?
My understanding is that LinearRegression is computing the closed form solution for linear regression (...
- 21
0
votes
2
answers
1k
views
How is the linear regression optimize in R and Python?
I am currently working a lot with R and Python. I am not able to access the C code the the R function lm_fit. I am wondering how is the linear regression optimize in R and python ?
I am pretty sure ...
- 301
1
vote
0
answers
3k
views
First order condition for OLS?
The below is quoted from this post: When is logistic regression solved in closed form?
"In OLS, you have
$$
\sum_i (y_i - x_i' \beta)^2 \to \min_\beta,
$$
which has the first order conditions
$$
-...
- 167
1
vote
1
answer
356
views
Computing overhead of statistical models for training?
Could someone provide overhead of the following model for training (With respect to input size or if there are any relevant parameters). Overhead I mean somewhat like asymptotic time complexity form.
...
- 1,467
2
votes
1
answer
262
views
Linear algebra use case [closed]
I learning some machine learning course, and I would like to know in which case we use linea algebra and Matrix Algebra?
Thank you
Kind regards
- 151