I always asked myself what was the right method name for a simple linear regression using running parameters.
I mean that instead of using constant mean $\bar{y}$ or $\bar{x}$ for the estimation of $ \alpha$ we use a simple moving average of $x$ and $y$ as well as a running standard deviation and a running correlation coefficient.
I've heard of least square moving average as well as local regression/running line but I'm not sure what is the correct formulation of such a method.
It sounds like you might be thinking of a Savitzky-Golay filter. This works by sliding a window across the signal. A local polynomial model is fit in each window using least squares. Local linear or quadratic models are the most common choices (but you could also think of a simple moving average as a 0th order Savitzky-Golay filter). Evaluating the model at the center of each window gives a smoothed version of the signal. It's also possible to differentiate the model to obtain smoothed derivatives.
When the samples are uniformly spaced, computation is very efficent--both smoothing and differentiation can be performed by convolving the signal with particular FIR filter coefficients, rather than by estimating each local model from scratch. But, to avoid edge effects in this case, it's necessary to iterate through points at the edges of the signal.
Related local polynomial regression techniques (e.g. LOESS, LOWESS) may also use weights to place greater emphasis on points near the center of the window.
As to the 'right method name for a simple linear regression using running parameters', I would suggest bivariate time series analysis of say univariate exponential moving averages (or autoregressive) time series models (see 'Example of Multivariate Time Series Analysis' here). Also, autoregressive models with a type of Markov switching in mean may be applicable. See historic review and comments in this paper.
Personally, I would put more trust in models like this,'Switching-regime regression for modeling and predicting a stock market return' described here.
