I was wondering, how are hilbert spaces and functional analysis useful to machine learning? I thought machine learning was a mix of statistics, computer science and optimization. How does functional analysis relate to that?
The whole theory of Reproducible Kernel System Space underlying the development of Support Vector Machines and Structured SVMs is built upon the theory of Hilbert spaces. Also the development of some applications of SVM like outlier detection, which is based on the idea of estimating the support of the unknown probability distribution (see Estimating the support of a high dimensional distribution, Schölkopf et al.).
Just to add to the answer of @SmallChess. In practice you can do without having a good knowledge of it, but you surely need to understand the implementations, the algebra involved, and geometrical interpretations of the solutions given by the algorithm of choice.