I am working on continuous evaluation of a regression model on streaming data from sensors. I think that Mean Absolute Error (MAE) can be found out iteratively similar to this link for averaging. $$ MAE_{t} = \left(\frac{N_t-1}{N_t}\right) MAE_{t-1} + \left(\frac{1}{N_t}\right) \left|y_t^{true}-y_t^{pred} \right| $$ I prefer Root Mean Square Error (RMSE) over MAE, since it penalizes more for higher error values (as mentioned in Mean absolute error OR root mean squared error?)
Is there an approach to find Root Mean Square Error (RMSE) in a similar way?
I thought of following ways:
Keep a big window of previous values and compute RMSE continuously whenever a new data arrives. (This is expensive both in time and space complexity)
Keep track of squared errors i.e. $$SqSum_{t} = SqSum_{t-1}+ \left(y_t^{true}-y_t^{pred} \right)^{2}$$ and find $$ RMSE_t= \sqrt{\frac{Sum_{t}}{N_t}}$$ In this approach, I am afraid that the squared sum may overflow the number limit for larger $N_t$.
Is there a better way to find RMSE?
I would be grateful if there are approaches which give more weight to the errors in the most recent predictions.