In logistics regression we use Gradient Descent to minimize the error function. For example:
We have error function
e(x) = hθ(x) - y
To get an appropriate θ to minimize error(x), we use Gradient Descent.
To simplify my question let`s assume θ, x, y are all normal numbers, not vectors.
So in Gradient Descent. First we get derivative of e(x)
Then we run below code in many loops until we get an appropriate θ. The step is a very small distance or shift on θ.
θ := θi - (∂e(x)/∂θ)* step
Here is what confused me.
(∂e(x)/∂θ)* step is a small shift on e(x) not θ. So why θ minus the shift of e(x) make sense? If θ need to minus something, I think the step is more appropriate since step is a shift of θ
Run θ := θi - step in a loop until that the (∂e(x)/∂θ) is zero can also get the right θ and it seems more reasonable