Linked Questions

0 votes
1 answer
500 views

Neural Network - Can I use sigmoid activation function in hidden layers of regression problem? [duplicate]

I am trying to predict for count which ranges from 0 onwards as a regression problem using NN. Can I add sigmoid, tanh or relu activation function to the hidden layers and no activation function to ...
jkjsdf fod's user avatar
0 votes
0 answers
237 views

What makes a neural network linear? [duplicate]

Two-part question: Neural Networks(NN) can be looked at as stacked units of logistic regression classifiers (LRC). A basic requirement of an activation function is to be non-linear. When LRC is a ...
Abhinav Mishra's user avatar
0 votes
0 answers
27 views

Neural Network Linear Activation Functions [duplicate]

I understand the intuition that the sum of linear functions is again linear, and that is why a neural network with linear activation functions yields a linear model. But what I'm confused about is ...
PerplexedPelican's user avatar
124 votes
7 answers
92k views

Why use gradient descent for linear regression, when a closed-form math solution is available?

I am taking the Machine Learning courses online and learnt about Gradient Descent for calculating the optimal values in the hypothesis. h(x) = B0 + B1X why we ...
Purus's user avatar
  • 1,343
40 votes
3 answers
16k views

Do we need gradient descent to find the coefficients of a linear regression model?

I was trying to learn machine learning using the Coursera material. In this lecture, Andrew Ng uses gradient descent algorithm to find the coefficients of the linear regression model that will ...
Victor's user avatar
  • 6,535
12 votes
1 answer
2k views

Why are activation functions needed in neural networks? [duplicate]

Why are activation functions needed in neural networks? I know that it is to capture "non-linearities", but I have never been able to find a proper down-to-earth explanation. In particular, I am ...
pir's user avatar
  • 5,056
3 votes
1 answer
3k views

What is meant by Expressiveness in neural network?

While studying Batch normalization, I came across the parameter sigma and beta in the output. And all the information said that they are added in order to retain the "expressive power of the ...
Dhiraj Dhakal's user avatar
1 vote
1 answer
403 views

Which function can approximated with Neural Networks using only linear activation functions?

I want to find out which functions can be approximated up to arbitrary accuracy using Neural Networks with only linear activations. On this page I found out that with linear activation functions, the ...
Leviathan's user avatar
  • 113
1 vote
0 answers
106 views

When I do not add an activation function to my convolutional layer the model gets quickly stuck in a local optima, why?

I have model A: ...
RogerKint's user avatar
  • 171
1 vote
1 answer
78 views

Why do we need stochastic gradient descent when we can precompute certain results in batch gradient descent?

I have read that batch gradient descent forces this summation at every step of the update, which makes it time consuming. But if we have the following hypothesis function: $$h(x^i) = w_0 + w_1x^i$$ ...
sage76's user avatar
  • 13