When forecasting, is it better to remove the outliers or just to transform them? I am forecasting the number of logins. I have a dataset with the number of logins for each hour.
First, I use LOF (local outlier factor) to find the outliers and then I remove them.
Second, I use Kalman filter to forecast.
But as you can see in the plot, the prediction (blue) is displaced in time from the logins (red) and also from the future confidence interval (green).
So, would it be better to transform the outliers (maybe make them "normal") rather than completely removing them? Because right now I have some gaps in time. For example (the first monday), I have logins , for each hour, from 00h to 15h and the from 18h until the next day. So there is a gap of 2 hours.
Thank you

 A: If the data are wrong, remove them regardless of whether or not they are outliers. If a data point is right but it is an outlier and you remove it, your analysis will be biased. This might even be the reason why your prediction desynchronizes with the trend. Transforming outliers is just a less extreme way of deleting them  that is also prone to bias.
The key point of the Kalman Filter is that it should be able to use the response history to inform the system state, but it doesn't seem that you are doing that. That means that at time $t$ you can use all of $t-1$ to generate predictions for $t$ and onward. Later, at time $t+k$ you can use all of $t+k-1$ to generate predictions for $t+k$ and onward.
So at the dotted line, are you showing your system forecasts for that time onward unconditional of the later response, and then superimposing the observed system state afterward? In that case, it's not surprising to see the forecast lose fidelity, that's the inevitable outcome: you just have to be clear about the target range for forecasts... nobody can predict the entire future in a statistical model.
The weights for the lagged response inputs are usually quite high, the best prediction of a system state is often the state it has most recently been in. When you run a forecast way out in the future, it is prone to desynchronizing. One way you can enforce a strictly seasonal adherence is to use the actual season as covariate inputs.
A: I would replace packets of contiguous missing values with hourly averages around the missing values. If the values are not missing but are anomalous either manually adjust them or estimate what they should have been via Intervention Detection which is essentially a forward prediction/fitted value for an anomaly.
Outliers represent effects/variables that are omitted from your model and if possible need to be identified and accounted for by adding additional/logical predictor series or worst case dummy indicators.
