I'm trying to understand boosting as gradient descent (GD) in functional space. I've followed the argument in the classic paper on the subject, but would characterize my understanding as tenuous at best. Proceeding by the way of analogy, I'll make following statements.
Regular GD aims to locate optimal points of scalar-valued function. Functional GD aims to find functions (over the entire domain of definition). In theory, regular GD is a numerical substitute for the calculus-based procedure of setting the derivative to zeros and solving the resulting equation (of course practical problems aren't easily solvable this way). Equivalent direct procedure in functional space is to find functional derivative and solve the resulting Euler-Lagrange equation. That is how we solve the isoperimetric problem, Brachistochrone problem, max entropy problems etc.
To further my understanding of functional GD, I want to solve a simple calculus of variations problem using functional GD. Say, I want to find a 2D curve that minimizes the distance between two points. I know how to solve this using Euler Lagrange. I want to reproduce this solution numerically by seeing a randomly picked function (obeying constraints) evolve toward a straight line.
Is there a text, a tutorial, arXiv document, blog, a video or any other document that provides a gentle introduction to functional GD using simple examples? My searches yielded some student-scribed lecture notes, but they didnt help much beyond clarifying the notation in the paper.
Can someone help set up a functional GD for a simple problem? I'm not asking for explicit code, as I'm quite happy to write that myself.