Linked Questions
21 questions linked to/from Why use gradient descent for linear regression, when a closed-form math solution is available?
2
votes
1
answer
143
views
Significant Difference in prediction when using library and coding from scratch in Multiple Linear Regression [duplicate]
I have been trying to implement multiple linear regression from scratch after implementing it using sklearn. The values predicted using sklearn is very accurate whereas the values predicted by the ...
- 465
1
vote
0
answers
46
views
Various Methods to Calculate Linear Regression [duplicate]
I have just started learning Machine Learning and one of the very first topics that I have encountered in this venture is Simple Linear Regression. From Andrew Ng's course, I have learned to perform ...
- 465
0
votes
0
answers
11
views
What's the point of using gradient descent for linear regression if you can calculate the coefficients directly using the least squares method? [duplicate]
Gradient descent involves significant computational effort, whereas the method of least squares enables direct and accurate calculation. Does gradient descent offer any advantages over least squares ...
- 33
20
votes
4
answers
16k
views
Why is gradient descent required?
When we can differentiate the cost function and find parameters by solving equations obtained through partial differentiation with respect to every parameter and find out where the cost function is ...
- 607
33
votes
3
answers
22k
views
How could stochastic gradient descent save time compared to standard gradient descent?
Standard Gradient Descent would compute gradient for the entire training dataset.
...
- 1,175
18
votes
3
answers
10k
views
Why not use the "normal equations" to find simple least squares coefficients?
I saw this list here and couldn't believe there were so many ways to solve least squares. The "normal equations" on Wikipedia seemed to be a fairly straight forward way:
$$
{\displaystyle {\begin{...
- 1,244
22
votes
2
answers
22k
views
How to choose the right optimization algorithm?
I need to find the minimum of a function. Reading the docs at http://docs.scipy.org/doc/scipy/reference/optimize.html I see that there are several algorithms that do the same thing, i.e. find the ...
- 2,077
10
votes
6
answers
6k
views
What is the purpose of a neural network activation function?
What is the purpose of a neural network having a non-linear activation function?
Is it correct to say that the non-linear activation function's main purpose is to allow the neural network's decision ...
- 3,630
7
votes
3
answers
9k
views
Linear regression for large dataset
If the dataset is too large to be entirely loaded into memory, how can we do linear regression with the dataset?
- 103
5
votes
3
answers
8k
views
Why use MSE instead of SSE as cost function in linear regression?
I am studying linear regression and I solved some problems analytically.
For that I used the normal and intuitive sum of squared error function.
Looking at this function, it makes all sense why it ...
- 301
10
votes
1
answer
6k
views
Why is optimisation solved with gradient descent rather than with an analytical solution? [duplicate]
I'm trying to understand why, when trying to minimise an objective function, gradient descent is often used, rather than setting the gradient of the error to zero, and solving it analytically.
In ...
- 7,129
4
votes
1
answer
5k
views
Single layer neural network as linear regression
I'm really struggling to see the analogy between linear regression and a single layer perceptron. They are supposedly the same thing.
I completely understand the concept of the inputs to the neuron ...
- 555
4
votes
1
answer
4k
views
Which ML Algorithms are affected by dummy variable trap?
My understanding is that regression models are affected by the dummy variable trap. What about other machine learning algorithms e.g. linear svm, logistic regression?
Also, if an algorithm is not ...
1
vote
2
answers
5k
views
Full-Rank design matrix from overdetermined linear model
I'm trying to create a full-rank design matrix X for a randomized block design model starting from something like the example from page 3/8 of this paper (Wayback Machine) .
It's been suggested that I ...
- 163
7
votes
1
answer
2k
views
Difference Between Linear Regression in Machine Learning and Statistical Model
I had the understanding that the major difference between machine learning and statistical model is, the later "assumes" certain type of distribution of data & based on that different model ...
- 6,466