'Outlier' is a convenient term for collecting data together that doesn't fit what you expect your process to look like, in order to remove from the analysis.
I would suggest never (caveat later) removing outliers. My background is statistical process control, so often deal with large volumes of automatically generated time-series data which is processed using a run chart / moving box plot / etc. depending on the data and distribution.
The thing with outliers is that they will always provide information about your 'process'. Often what you are thinking of as one process is actually many processes and it is far more complex than you give it credit for.
Using the example in your question, I would suggest there could be a number of 'processes'. there will be variation due to ...
- samples taken by one conductance device
- samples taken between conductance devices
- when the subject removed a probe
- when the subject moved
- differences within one subject's skin across their body or between different sampling days (hair, moisture, oil, etc)
- differences between subjects
- the training of the person taking the measurements and variations between staff
All of these processes will produce extra variation in the data and will probably move the mean and change the shape of the distribution. Many of these you won't be able to separate into distinct processes.
So going to the idea of removing data points as 'outliers' ... I would only remove data points, when I can definitely attribute them to a particular 'process' that I want to not include in my analysis. You then need to ensure that the reasons for non inclusion are recorded as part of your analysis, so it is obvious. Don't assume attribution, that's the key thing about taking extra notes through observation during your data collection.
I would challenge your statement 'because most of them are errors anyway', as they are not errors, but just part of a different process that you have identified within your measurements as being different.
In your example, I think it is reasonable to exclude data points that you can attribute to a separate process that you don't want to analyse.