I had a matrix with values and I wanted to identify the outliers. For doing so, I computed the z-scores (so, I had a new matrix with z-scores) and I took those values above 3.

Now, I have 2 replicates (2 matrices from the same origin). Therefore, I would like to consider both matrices in order to get real outliers for the origin it comes from and to exclude those replicate-specific outliers.

Do you have any idea?

  • 1
    $\begingroup$ What do the rows and columns of the matrix represent? $\endgroup$ – shadowtalker Jun 27 '17 at 13:17
  • $\begingroup$ @ssdecontrol counts $\endgroup$ – user2886545 Jun 28 '17 at 8:24

Outliers are rather tricky beasts that appear and disappear in our experiments.

Let's consider the facts: we are analyzing a population of "occurrences" by running an experiment and taking samples from said population.

We have chosen a certain set of variables as part of the design of said experiment.

While running some of the sets of captures, some of the individuals of some samples present what seems like outlier values.

Is it the result of a measurement error?

Is it the result of a error in the process of data capture, data registration, or data transcription?

Is it that we are using the wrong set of variables?

Is it that the samples are too small, or the data captures are not evenly spaced in time, or maybe some other factor not properly considered by the design of the experiment?

The point is, there could be a few open fronts in the design of said experiment.

To figure out outliers, we need to consider a set of strategies based on different approaches to analyze them, and then produce a hierarchy of results to determine if any given individual is really an outlier or is not.

In these strategies based on different approaches, we should always include cluster analysis, to validate how individuals are grouped (are groups changing over time?), and discriminant analysis, PCA and Factor Analysis to check to which group or groups belong each and every given individual of a sample, but we should always strive to keep our options open and flexible.

Do we really understand the main forces that drive the behaviour of the occurrences that we are capturing?

Is the design of the experiment giving us the proper insights into these forces and how they drive and control the behaviour of the occurrences?

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.