Do I want to overfit, when doing outlier detection based on regression? Imagine, we have speed data of car and we would like to detect, if car speeds up or down more than it should.
Do I want to just overfit my model, so the outlier (higher or lower speed) would lead me to higher residual easing the process of outlier detection?
Edit: To clarify, training data have no outliers.
 A: 
Do I want to just overfit my model, so the outlier (higher or lower speed) would lead me to higher residual easing the process of outlier detection?

No, you don't. A heavily overfitted model would perfectly fit your data, but not generalize outside it. So it can be the case that it would consider as an outlier anything that is not alike to your training data. For an overfitted model to work for outlier detection, your training data would need to contain every possible scenario to be observed in the real world. But, if you had data on every possible scenario, you wouldn't need a machine learning model, since you could simply search your database and if the datapoint is not found among the "valid" cases, it is an anomaly. Most likely, your data does not contain every possible scenario, especially since speed is a real number so there is an infinite number of them. That is why you need a model that is able to fill in the gaps, so it needs to generalize outside of the training data.
A: If your goal is to detect "if car speeds up or down more than it should", then your regression model should capture the kinds of acceleration the car "should" be performing, but not capture the kinds it "shouldn't". If your model is very over-fit, it will capture all kinds of acceleration, and nothing will be an outlier. If it's under-fit, it won't capture some acceleration that you think of as normal, so will show too many outliers.
All that aside, linear regression on it's own is not generally appropriate for time series data, and not necessarily the best way of identifying outliers in any kind of data. There are specific approaches for identifying outliers in time series data you should look into.
