So I’ve seen the code of an R package where a two dimensional optimisation (actually MLE, finding the minimum of the negative log likelihood) is performed with the optim function and also two optimise functions to find the initial values for that optim function. That is, let $F(x,y)$ be the function whose min to be sought, first fixed $y=y_0$ (some constant) and find $x$ with optimise. Then $x$ is fixed at this optimised value and find $y$ with optimise. If, on the top of these two optimise along the $x$ and $y$ axis, can the initial values of optim be further improved by optimising along, say, diagonally with the use of directional derivative?
Yes, steepest gradient descent is a popular method that improve its solution by moving along the steepest descent direction.
Also, you can for instance be creative and let it move along other directions and solve a small optimization problem locally. However, unless the direction that you pick has an advantage (For example, some eigenvector direction or conjugate graidient direction), I don't see the benefits of picking a random direction.