I have the following issue. I want to perform clustering based anomaly detection on high dimensional timeseries data. To be able to do so I'm performing PCA before clustering, and before that I'm performing standardization of my parameters.
Here is the problem - a number of parameters has a lot of spikes, but these are not outliers, so I cannot remove them just like that. As this is a history of a long (more than a month of recording) time serie it contains a lot more of "flat" data than the spikes. Hence, the spikes act as outliers as there are not that many of them. This causes normalization to mess-up things a bit, as some of the parameters have spikes while others do not. This means that standardization basically "fails" due to the spikes in certain parameters, causing my standardized parameters to have different scale again.
As the normalization is not good, neither is the PCA and clustering of course suffers from parameters on different scale so clusters are hidden due to parameters on higher scale which act as "boosted".
How do I overcome this problem? What is the procedure to make them on more or less same scale when some have these spikes during time?
I should add that at the end I'm trying to find anomalies by clustering data, hopping for either small clusters or "points" which are far from cluster representatives.