An outlier is an observation that appears to be unusual or not well described relative to a simple characterization of a dataset. A discomfiting possibility is that these data come from a different population than the one intended to be studied.

learn more… | top users | synonyms (1)

0
votes
0answers
17 views

Robust one sample tests of variance or scale

A common one sample test for variance is the chi-square test, e.g., http://www.itl.nist.gov/div898/handbook/eda/section3/eda358.htm. What are some robust testing alternatives for variance when the ...
0
votes
0answers
9 views

Do generative and discriminative models have different robustness against outliers?

I am wondering whether some general distinction can be made about how discriminative and generative models handle outliers.
0
votes
0answers
13 views

How to perform outlier detection in a multi class dataset?

I have a dataset with 40 class labels and outlier data in it. The dataset is a high dimensional one with 1500 features. I am in need of building a model to detect the outliers in the dataset, using ...
1
vote
1answer
30 views

Boxplot Outliers

I'm looking for outliers in a non-normally distributed dataset: n: 1,900 Mean: 2,738 StDev: 1,544 Min: 1 Max: 22,102 Anderson-darling: 40 P < 0.005 The boxplot shows the outliers in one ...
0
votes
0answers
4 views

how to find influential data point in an aggregate

In an explanatory study (and not a predictive one), I want to find the influence of an extreme point (from a data set) on an aggregate (may be sum of the all the data points in the set)? I have found ...
0
votes
0answers
18 views

removing outliers in poisson distributed data (count data) to improve classifier accuracy [on hold]

I wanted to know how to remove outliers in count data, the data i have is labeled can I remove the outliers from each class ? Or consider only 80% of the data close to the cluster center to train the ...
2
votes
0answers
19 views

Changing sensitivity (cval) in tsoutliers resulting in unexpected results

I am using the excellent tsoutliers R package to detect outliers (additive outliers, temporary changes etc.), but the cval ...
0
votes
0answers
6 views

Calibrate sensors to each other?

I have 40 temperature sensors that have been in a controlled environment (over 40°C range) they are all factory Calibrated to 0.5°C for room temperature. What would be a good statistical method to ...
0
votes
0answers
7 views

Robuste classification of ground and no-ground points in dense forest LiDAR Airborne

This is my first project on my graduation so take easy. Im studying methods to remove outliers on LiDAR airbone data in the tropical amazon rainforest. Remove outliers above the trees is easy, the ...
1
vote
1answer
20 views

what is the appropriate one-class classifier for sequenced categorical data?

Can someone suggest an algorithm for an outlier detection system? My requirements are: it supposed to be a one class classifier, where on training phase, it only fed 'normal' data however the normal ...
0
votes
0answers
7 views

Question about MCD and R's cov() function

I'm trying to identify outliers using mahalanobis distance acquired by the covariance matrix determined by the MCD-selected observations of a data matrix X (i.e. the vectors contained within the ...
0
votes
0answers
10 views

Comparison of Stahel-Donoho and Minimum Covariance Determinant estimation

I am interested in detecting multivariate outliers in a low dimensional data set ($n<p$). Various high-breakdown robust methods for multivariate settings such as Stahel-Donoho and Minimum ...
0
votes
0answers
13 views

usage of adjbox() in R [duplicate]

Anyone here who could describe the usage of adjbox()? Is it used for very skewed distributions to have a more generous range like the boxplot? What is the formula like mean+1.5*IQR in boxplots? Is it ...
0
votes
0answers
14 views

chisquare outlier test assumptions

what are the assumptions of the chisquare outlier test? ( in R: chisq.out.test()) Is it only applicable if the data follow a certain Distribution? What is the idea of ist Definition of outliers? ...
0
votes
0answers
33 views

Identify outlier usage intervals in time-series data

I want to find outliers in power consumption in real-time, at hourly rate, i.e., at the end of the hour, I should say whether power consumption in current hour was outlier/anomalous or not. Approach: ...
0
votes
0answers
21 views

Appropriate testing for outliers

I want to perform an outlier-test for my data. The distribution of the data looks like a hyperbola (there are many values near 0, all values are >0). Sample size of data is 5000. I want to find an ...
0
votes
0answers
9 views

Finding outliers on LiDAR for forests

I'm working with LiDAR data in college and focusing on preprocessing. I'm still graduating and this is my first project. So, I used the library of c++, Point Cloud Library (PCL) to deal with the ...
2
votes
0answers
33 views

Dealing with noisy/mislabelled dataset

I have several datasets where each instance has numeric label assigned by a human that can take values between 1 and 5. After doing a manual inspection of one of these datasets, I noticed the ...
0
votes
1answer
27 views

Finding outlier values for non-normally distributed data

I have univariate data (38 is the sample size).The distribution is certainly not normal. How can I find the outliers? I used z-score but am not getting a desired result.
1
vote
0answers
18 views

Transformation and/or Winsorizing?

I want to compare two group of 24 and 28 people with t-test on type of activity (5 different's types of activity and a total), later one the same value will be use in regression logistic. If you ...
0
votes
0answers
32 views

PROC ROBUSTREG or PROC NLMIXED in SAS, to down-weight effects of non-normality and outliers

We are conducting an OLS regression in SAS. We have performed all diagnostic tests and concluded that there is non-normality in the residuals, influential outliers and heteroskedasticity. We would now ...
0
votes
2answers
55 views

How do I remove outliers in dataset?

I have a data-set (185 rows) with 20 predictors and 1 dependent variable. I have applied Cook's distance and then 4/N formula to remove some of the outliers in 1st iteration. Should I do this ...
3
votes
1answer
34 views

Do points with high Cook's distance necessarily have a high standardized residual, and vice-versa?

I have two questions below: Could a data point be an influential point if its cook distance is outstanding(greater than 4/(n-p-1)) while its standardised residual is less than 2? It looks like to me ...
0
votes
0answers
19 views

With or without outlier and/or parametric vs non-paramteric

I'm comparing two group on the number or movie seen in the last mont (30 people in each group). Spss say that there 4 outlier but when I'm lookin with the z score I only find 1. If I do a t-test it'...
1
vote
0answers
24 views

Winsorizing, just the outlier or all the value?

I have an outlier in my data set. I want to use the winsorizing quartile (to change the outlier to the 5th% and/or 95th%). Looking at the quartiles, sometimes I have more values than just my ...
1
vote
1answer
31 views

Outlier transformation for t-test

In a normally distributed sample with one outlier, some suggest changing the value of the outlier to be one unit above the next highest score. If you are doing a comparison test on two groups using ...
1
vote
1answer
37 views

Robust statistic for representing small dataset with outliers and representing them graphically

I'm evaluating different systems varying a certain parameter common to all of them. Let this parameter be x. At each x value I ...
0
votes
0answers
6 views

Using historic water flux data to detect the existence of leakage, where should I start?

This question is also linked to How to detect a relatively small level shift(leakage) in an hourly water flux time series in an area? which I asked a week ago... Background I've got a series of ...
0
votes
0answers
8 views

Replacing an outlier in one of two terms with a mean

Should I replace values with means? I have an experiment and I know (from the literature) that 3000ms reaction times means the participant is not attentive (the outliers reach 20000ms). I'm analyzing ...
1
vote
1answer
46 views

What comes first: outlier detection or model selection?

I'm fitting a GLMM (mixed logistic regression) in R. I have five covariates. For model selection, I'm using glmmLasso() (in R) to determine which of the five covariates and their interactions should ...
0
votes
1answer
42 views

Finding outliers in multiple dimensions

I'm working on dataset which isn't normally distributed. It contains three dimensions: cost, discount and profit. I'm trying to find outliers in all these dimensions. I used $\text{z-score}$ to find ...
0
votes
1answer
27 views

picking out outliers from a GLM in R

I recently fitted a beta distributed GLM using R (and the betareg package). as you can see the model is a reasonably decent fit, however there are a few outliers. i would like to run the model again ...
0
votes
0answers
28 views

Why leverage measure the distance of the ith observation from the center of the x space? [duplicate]

I know the definition of leverage points in regression, that is $h_{ii}=x_{i}'(X'X)^{-1}x_{i}. $ In many places and text books, they always say that leverage is a standardized measure of the distance ...
3
votes
0answers
21 views

How to detect a relatively small level shift(leakage) in an hourly water flux time series in an area?

Background I'm working on a project which aims to use the history data about a water flux to detect whether there is a leakage happened. The data is hourly collected and among about 4 months. I've ...
0
votes
0answers
38 views

Why different diagnostic tests for detecting outliers in linear regression don't agree with each other?

I have one predictor and one variable, which means that the "cutoff" Mahalanobis value is > 3.84 at 0.05 significance level; covariance ratio should be between 0.88 and 1.12 (1+-3x(1+1)/50); the "...
1
vote
0answers
7 views

Are there kernel-based one-class sparse kernel-based outlier detection methods, e.g. one-class Relevance Vector Machine?

I have a commercial outlier detection problem in moderate dimension (8-25). We have a limited number of true positive tags and can roughly evaluate performance of various methods. So far, the 1-...
0
votes
0answers
6 views

Intuitive explanation of Grubb's test

Can anyone give an explanation of Grubb's test for outliers? I've found many resources giving steps to calculate it, but none even attempting to talk about why that equation should be a good outlier ...
0
votes
0answers
25 views

How to deal with outliers and feature selection simultaneously?

I've been given some data and need to pick what I consider to be the best features from it and use them to build models that fit the data. My issue is that all the tests I've seen for outliers assume ...
0
votes
1answer
49 views

anomaly/outliers detection in a multilabel dataset on the outcomes

Assuming a multilabel dataset contains a few wrong data. If so, is there a way to predict those wrong outcome given the fact there is a 'pattern' in the predictors? Let's use 'baby and silly' example:...
0
votes
0answers
27 views

Outliers detections in time-series

I am searching algorithms for detecting outliers in a time-series data. I see that there are a lot of algorithms and they have an implementation in R. But i don't find any explanation on how they work....
0
votes
0answers
15 views

Which Statistical model should I Choose to fit the Data?

I am struggling with this problem for quite a few days.So far, I have used simple Box-Plot method to pick out the outliers for each location and Diseases. And how can I get outliers after fitting any ...
2
votes
0answers
9 views

Check for outlier in random effects

I have 3 treatements and each treatment is given to 2 random people from which 2 samples is taken from each person that gets the specific treatment. The dataframe in R below shows a sample dataset ...
0
votes
0answers
32 views

Testing for discords in seasonal time series data

I'm trying to find a way to detect discords in seasonal data. I have an algorithm that can select the most likely sub-sequence to be a discord, but what I'm missing is an actual test. I know that ...
1
vote
0answers
33 views

Outlier transformation or no transformation?

I have two group's of 25 people each. I want to know if there a difference between them on a past experience of happiness . I have a test with 100 question in it and all question have an a and b ...
1
vote
0answers
42 views

Linear Regression – When is Bad Data “Too Much”?

I am doing a multiple linear regression analysis. One variable, which I think may be quite predictive, has known bad data. I am currently sampling and using analysts to independently verify the ...
5
votes
1answer
49 views

Cook's distance in detecting outliers

According to my understanding, Cook's distance measures the influence of each observation by excluding points when fitting a model. So I assume it could be an reasonable approach for outlier detection?...
0
votes
0answers
20 views

Outliers detection or some roubust metrics on long tail sqewed distributions

I have a distribution of user sessions on the web site in the following format date,sessionId,price 2010-01-01,1,0 2010-01-01,2,0 2010-01-01,3,10 ... And I am ...