Pointwise rates of convergence What pointwise rates of convergence exist for nonparametric estimation?
For example, I have found that for kernel regression, 
$$
|m_n(x) - m(x)| = O(n^{((s-1)/s}h^d)^{-1/2})
$$
where $s$ is the largest moment that exists and $d$ is the dimension. (Krzyzak and Pawlak, 1987). 
Do results like this exist for least squares estimation, LASSO, k-NN, etc.?
 A: Your question queries the rate of convergence of non-parametric estimation for OLS LASSO, etc. There are multiple assumptions implied by this question that should be mentioned. The first assumption is the existence of solutions to which an algorithm can converge. The second is whether or not our regression algorithm is convergent, and what has to be done to insure convergence. Only then do we have a context that is general enough to discuss convergence rates.
The existence of solutions: Suppose we have only 4 samples and a 4 parameter model consisting of a mixture distribution of two exponential distributions. This is not regression in the sense of being overdefined, and we selected that to make a point. An exact solution may be undefined in real space. HOWEVER, there is always one or more (Indeed N-tuple) exact solutions if we allow for the complex field, and regress using the complex variable form of the biexponential mixture distribution (Yes, there is such a thing). Why mention this? 
The existence of convergent solutions: The point is that
There is always at least one convergent solution. However, it is up to you to find it, and it may be nonphysical, complex-valued, ridiculous and useless. 
Taking ordinary least squares (OLS) and for the purpose of discussion, it does not matter whether this is applied to non-parametric data or parametric data.
There are many regression methods that can be applied to OLS. Some have global convergence, some do not. Now, random search is global and very slowly convergent and Nelder-Mead converges a lot faster but is not as global, e.g., see constrained global non-linear regression, which lists multiple regression methods all of which have different convergence rates. These rates from fastest to slowest (but only for the ones listed by Mathematica) are Nelder-Mead, differential evolution, simulated annealing and random search. In that list is simulated annealing, which will always converge. However, it often converges to an approximate solution as opposed to an exact one.
There are undoubtedly method for which the OP's question can be answered in exact form, however, these can be stated only in the context of 1) having solutions and 2) having algorithms that converge to those exact solutions and in practice those problems determine which regression method can be used. For example, for typical data from drug concentrations, the biexponential distribution mixture model mentioned above is so poorly conditioned that random search converges faster in fewer steps than Nelder-Mead. 
Summarizing, there is are tendencies as to how quickly algorithms will converge, but in any one case there is too much variability in how regression algorithms, fit models, and data interact to allow for a priori selection of an algorithm, fit equation form and appropriate data range to assure 1) existence of a solution 2) convergence to a solution and finally 3) rate of convergence.
There is one final caveat. The conditions for non-parametric fitting may have certain advantages over fitting parametric data. These pertain to better normality, and nicer conditions for convergence, and may mitigate some of the problems encountered in parametric regression. Nevertheless, one should be skeptical about statements pertaining to rates of convergence as much can go wrong and there are a lot of preconditions that have to be met.    
