# Linked Questions

6answers
30k views

### What algorithm is used in linear regression?

I usually hear about "ordinary least squares". Is that the most widely used algorithm used for linear regression? Are there reasons to use a different one?
2answers
12k views

### Do we need gradient descent to find the coefficients of a linear regression model?

I was trying to learn machine learning using the Coursera material. In this lecture, Andrew Ng uses gradient descent algorithm to find the coefficients of the linear regression model that will ...
2answers
13k views

### How could stochastic gradient descent save time compared to standard gradient descent?

Standard Gradient Descent would compute gradient for the entire training dataset. ...
3answers
7k views

### Why not use the “normal equations” to find simple least squares coefficients?

I saw this list here and couldn't believe there were so many ways to solve least squares. The "normal equations" on Wikipedia seemed to be a fairly straight forward way:  {\displaystyle {\begin{...
2answers
17k views

### How to choose the right optimization algorithm?

I need to find the minimum of a function. Reading the docs at http://docs.scipy.org/doc/scipy/reference/optimize.html I see that there are several algorithms that do the same thing, i.e. find the ...
4answers
11k views

### Why is gradient descent required?

When we can differentiate the cost function and find parameters by solving equations obtained through partial differentiation with respect to every parameter and find out where the cost function is ...
1answer
4k views

### Why is optimisation solved with gradient descent rather than with an analytical solution? [duplicate]

I'm trying to understand why, when trying to minimise an objective function, gradient descent is often used, rather than setting the gradient of the error to zero, and solving it analytically. In ...
5answers
5k views

### What is the purpose of a neural network activation function?

What is the purpose of a neural network having a non-linear activation function? Is it correct to say that the non-linear activation function's main purpose is to allow the neural network's decision ...
1answer
974 views

### Difference Between Linear Regression in Machine Learning and Statistical Model

I had the understanding that the major difference between machine learning and statistical model is, the later "assumes" certain type of distribution of data & based on that different model ...
3answers
4k views

### Linear regression for large dataset

If the dataset is too large to be entirely loaded into memory, how can we do linear regression with the dataset?
1answer
46 views

### Significant Difference in prediction when using library and coding from scratch in Multiple Linear Regression [duplicate]

I have been trying to implement multiple linear regression from scratch after implementing it using sklearn. The values predicted using sklearn is very accurate whereas the values predicted by the ...
1answer
1k views

### Which ML Algorithms are affected by dummy variable trap?

My understanding is that regression models are affected by the dummy variable trap. What about other machine learning algorithms e.g. linear svm, logistic regression? Also, if an algorithm is not ...
2answers
2k views

### sklearn Linear Regression vs Batch Gradient Descent

tldr: Why would sklearn LinearRegression give a different result than gradient descent? My understanding is that LinearRegression is computing the closed form solution for linear regression (...
2answers
2k views

### Full-Rank design matrix from overdetermined linear model

I'm trying to create a full-rank design matrix X for a randomized block design model starting from something like the example from page 3/8 of this paper . It's been suggested that I can go about ...
1answer
56 views

### Is stochastic gradient descent unable to learn more complex models which batch gradient descent can learn?

Here's a toy dataset. ...
0answers
35 views

### Various Methods to Calculate Linear Regression [duplicate]

I have just started learning Machine Learning and one of the very first topics that I have encountered in this venture is Simple Linear Regression. From Andrew Ng's course, I have learned to perform ...
0answers
56 views

### Linear least squares algorithms

I have stumbled across these two questions and accepted answers: (1) Do we need gradient descent to find the coefficients of a linear regression model? (2) Why use gradient descent for linear ...
0answers
404 views

### Linear Regression using Matrices vs. Covariance(x,y) / Var(x)

From taking some online linear regression courses (Stanford Machine Learning and John Hopkins Linear Regression) it seems that there are at least three ways of finding a the coefficients of a linear ...
1answer
73 views

### The best line fit can be found analytically by the least squares method. So can we say that linear regression (least squares) has an optimizer?

The best line fit can be found analytically by the least squares method. So can we say that linear regression (least squares) has an optimizer? For example, for logistic regression I can use an ...
2answers
1k views

### How is the linear regression optimize in R and Python?

I am currently working a lot with R and Python. I am not able to access the C code the the R function lm_fit. I am wondering how is the linear regression optimize in R and python ? I am pretty sure ...