While reading the implicit stochastic gradient descent, I get stuck by its update:

$$w^{new}:=w^{old} - \eta \nabla Q_{i} (w^{new}) $$

I learned from this slide that the above is the implicit style of SGD which can be written as this:

$$\theta_{k+1} = \theta_k - \gamma_k \nabla_k f_i(\theta_{k+1})$$

But I wonder how we can use $\theta_{k+1}$ before even defining it? How does it differ from the explicit stochastic gradient descent in implementation? Why do we need implicit SGD? A simple example for illustration would be very appreciated. Thanks in advance.

While reading the implicit stochastic gradient descent, I get stuck by its update:

$$w^{new}:=w^{old} - \eta \nabla Q_{i} (w^{new}) $$

I learned from this slide that the above is the implicit style of SGD which can be written as this:

$$\theta_{k+1} = \theta_k - \gamma_k \nabla_k f_i(\theta_{k+1})$$

But I wonder how we can use $\theta_{k+1}$ before even defining it? How does it differ from the explicit stochastic gradient descent in implementation? Why do we need implicit SGD? A simple example for illustration would be very appreciated.