# How to choose the learning rate for stochastic gradient descent (via backtracking)?

I am trying to implement "from scratch" SGD and Mini Batch Gradient Descent in Matlab.

I have to minimize a function like $$f(x)= \sum f_i (X_i, y_i)$$ where $$(X_i, y_i)$$ is a data point (features and class label one hot encoded respectively)

However, I am a bit confused about how to choose the learning rate. When implementing gradient descent, I used backtracking line search in order to find a learning rate that always guarantees a descent direction.

However, in SGD (or even in MBGD), one choose randomly some $$k$$ data points ($$k=1$$ for SGD). Let's call them $$x_{i_{l_1}},..., x_{i_{l_k}}$$. Let's call those indexes $$l_1,...l_k$$ the subset $$L$$. The gradient is now only taken on $$\sum_{i\in L} f_i (X_i, y_i)$$. The direction given by the gradient of this "partial" function is not necessarily a descent direction, thus how to find a "good" learning rate ?

I guess that backtracking taking into account the overall objective function is not that useful, as there is actually no guarantee to obtain a learning rate that could guarantee the decrease of the original $$f(x)$$... So I was thinking about making backtracking but only on the batch of points $$L$$ used for computing the gradient (i.e. find a good learning rate that minimizes $$\sum_{i\in L} f_i (X_i, y_i)$$ rather than the overall function $$f(x)$$).

Does it make sense ?

Hoping to get an answer soon !