Tagged Questions

0
votes
0answers
38 views

Removing data above the mean

I am working with a data set, for which to remove noisy data I take the average of the sample itself and cut anything above that average. What I'm trying to understand here is does this have a name ...
1
vote
2answers
152 views

Is this outlier merely unusual or does it signify manipulation?

These are actual reported costs (60) for a series of projects for two years, say USD (though actually not) totalling $57,975,403.24: ...
1
vote
6answers
699 views

How to detect outliers in skewed data set?

I am working on my school datamining project. Within preprocessing stage I need to remove outliers from my data set which is positively skewed (see description). I have an idea to remove all values ...
1
vote
1answer
94 views

Dataset and outlier question

I'm facing a data dilemma. I would like to have a real data illustration for an outlier detection rule i'm working on. The outlier detection rule targets datasets of continuous (not necessarily ...
10
votes
3answers
304 views

Good books covering data preprocessing and outlier detection techniques

As the title goes, does anyone know of a good, up to date book that covers data preprocessing in general and especially outlier detection techniques? The book doesn't need to be focusing exclusively ...
49
votes
5answers
1k views

Essential data checking tests

In my job role I often work with other people's datasets, non-experts bring me clinical data and I help them to summarise it and perform statistical tests. The problem I am having is that the ...
5
votes
1answer
220 views

Weird residuals in linear regression

I analyze a set of multivariate measurements. It is known that several pairs of independent variables show high linear correlation. The graph below shows a scatterplot of one such pair (X and Y, upper ...
9
votes
4answers
547 views

Separating two populations from the sample

I'm trying to separate two groups of values from a single data set. I can assume that one of the populations is normally distributed and is at least half the size of the sample. The values of the ...