How to handle timeseries extremes (sigma > 20) in deep learning?

Question

I'm using 16-channel, 400-Hz, standardized EEG data to train CNN-LSTM for seizure classification. The data contains $O(3)$ sigma > 20 points, rarely thousands in a row - and $O(4)$ sigma > 10 points (imgs below). Overall, they comprise a very small fraction of all data - regardless, I'm unsure of their impact on model's learning.

I've inadvertently left in one sample in a batch of 32 as nonstandardized, having it end up with sigma=52 - which almost always severely disrupted the model, causing gradient death, and classifier flip-flops (always predicting '0', then always predicting '1', etc). BatchNormalization layers looked completely different in both train and inference modes. This suggests a notable fraction of a single sample having extreme points may notably harm training. Also, such a fraction pre-standardization can, and did, substantially shrink all other points in the sample. Lastly, the data itself is strongly Normally-distributed (img below).

All considered, what's a reliable remedy? Ideas so far:

• Clipping - but at what value to clip?
• Noisy clipping - e.g. clip at 10, subtract random uniform $\sim(0,1)$

---

Answer 1

I would start by determining whether those are are "real" brain signals or noise.

I don't know much about your equipment (and it's not particularly on-topic here), but I associate that exponential shape with an amplifier recovering from saturation. This would be consistent with the signal being much, much larger than expected, so my first guess would be a motion artifact. Some EEG systems have built-in accelerometers, which might help you check that (and could provide a useful feature for cleaning the data--or even detecting seizures).

At a minimum, you should be able to look up the levels at which your system clips or becomes distorted. You might want to consider introducing a third class ("noise" or "uninterpretable") for segments that approach those limits.