An outlier is an observation that appears to be unusual or not well described relative to a simple characterization of a dataset. A discomfiting possibility is that these data come from a different population than the one intended to be studied.

learn more… | top users | synonyms (1)

0
votes
0answers
7 views

outlier detection : use trimmed data to calculate STD?

I can use box-plot to examine data and detect outliers based on box-plot. Most often Box-plot uses a q-range (q3-q1) to define the data boundary and then display "outliers". Sometimes I see people ...
0
votes
0answers
10 views

Outlier detection in GARCH(1,1) in R by Doornik & Ooms (2002)

I try to find additive and innovative outliers in the German Stock Index (DAX) using the method Doornik & Ooms explained in 2002: Estimate the baseline GARCH model to obtain log-likelihood ($lb$)...
0
votes
0answers
9 views

How can I get lid out outliers in Repeated Measures ANOVA?

I'm a person who has barely statistic knowledge. Anyway, I'm analyzing my repeated measures data. Firstly, I did repeated measures ANOVA using "ez" package in R. However, the results don't have ...
0
votes
0answers
10 views

Outliers detection methods for RNA-Seq data

I have a rna-seq dataset with normalization in RPKM. The dataset have 1 gene per row with 4 different experiment condition. I need a detect de outliers values in this dataset. I used de weka filter ...
1
vote
0answers
71 views
+50

Robust M-estimator of location and dispersion by hand in R

I try to estimate a location and dispersion model with R, as described Maronna et al (2006, pp. 56). However, my estimate dispersion does not converge to the desired value. Do I have an error in the ...
0
votes
0answers
15 views

winsorization - reducing the effect of outliers

I have measured the response times each participant took to respond to 24 items, however, only times of the correct responses of each participant were considered, thus leaving me with distinct number ...
1
vote
2answers
53 views

Should I remove the outlier?

I want to run an ANOVA test. I am therefore testing for normality. I have tested each group and the residuals (group together)for normality. My data sample does not look approximately normal. However ...
0
votes
0answers
7 views

measuring errors of bias, dispersion and outlier rate

I fit different models to a sample of data using Bayesian statistics. I have obtained for each data point in the sample a posterior probability distribution. Assuming I know the true answers for the ...
0
votes
0answers
21 views

Simulate different types of outliers (with R) in a linear regression?

I'm trying to simulate a regression model with outliers to implement and understand more deeply the robust regression. I tried using a mixture between normal errors and uniforms.But as you can see, ...
0
votes
0answers
13 views

Find similarity or dependency of attributes in a high dimensional dataset [closed]

I have a huge dataset with around 25 variables. All attributes are numerical (continuous) in nature. I want to find the dependency and the structure among these variables. Also what can be the best ...
3
votes
1answer
48 views

Should outliers be removed from Principal Components Analysis?

I have calculated Hotelling's T2 statistic for detection of outliers in PCA analysis in Matlab. However, I am unsure as to whether or not it is a robust approach to remove these outliers? The output ...
3
votes
0answers
42 views

R package for classification and outlier detection together

I have a similar problem as this one. My training samples contain N observations and K>2 classes. I want to classify my test samples into one of the K classes, or as an outlier if it is far from any ...
0
votes
0answers
18 views

Real world practical Outliers/Anomalies

I want to have some real life and practical examples for anomalies that can be expressed like a scenario. For example in a country following left-hand driving, a vehicle moving in a wrong direction ...
0
votes
0answers
15 views

Suggestions for handling outliers

I'm comparing a duration measure (days) for five groups, each of 200 cases. For four of the groups the mean is about 4, standard deviation is about 3, and range is about 1-30. Group 5 is similar ...
-1
votes
1answer
32 views

Test for normality with outliers produces strange p-values

I try to create some example that show how an outlier causes non-normality. Therefore I created two datasets: A dataset with normal distributed data ...
2
votes
2answers
76 views

what to do when outliers are identified?

In one task that measures the times that participants take to respond to each item of a task some of the response times were considered outliers as they are more than 3SD above the mean of the ...
0
votes
0answers
27 views

Outlier/Anomaly detection in strings

Can anyone suggest methods/techniques to find anomaly in string data from database. The data contains road names, so every cell is unique. By outlier, I mean the strings with weird stuff(special ...
0
votes
2answers
38 views

How to interpret whiskers of a box plot when there are outliers?

When I look at the definition of box plots, the whiskers are said to indicate the extreme values. So, if I look at height in a sample of people, the lower whisker should denote the height of the ...
0
votes
0answers
18 views

Boxplot for detecting outliers… does it make sense?

At the moment I'm writing my master thesis. For that, I need to do some statistics based on my data, which I received from my study. In my thesis, I would like to analyze 4 different visualization ...
1
vote
1answer
31 views

Represent Email data with respect to time

I have a bunch of problem tickets that have activity logs in the form of Email-Outbound and Email-Inbound. Outbound is an email sent and inbound is an email received. I want to identify the pain ...
1
vote
4answers
93 views

How can I identify and remove outliers in R

I am performing regression analysis on prices of product that we have purchased, based on size and other attributes. However there are often buys in odd circumstances which factor into the price, ...
0
votes
0answers
22 views

Robust one sample tests of variance or scale

A common one sample test for variance is the chi-square test, e.g., http://www.itl.nist.gov/div898/handbook/eda/section3/eda358.htm. What are some robust testing alternatives for variance when the ...
0
votes
0answers
9 views

Do generative and discriminative models have different robustness against outliers?

I am wondering whether some general distinction can be made about how discriminative and generative models handle outliers.
0
votes
0answers
17 views

How to perform outlier detection in a multi class dataset?

I have a dataset with 40 class labels and outlier data in it. The dataset is a high dimensional one with 1500 features. I am in need of building a model to detect the outliers in the dataset, using ...
1
vote
1answer
37 views

Boxplot Outliers

I'm looking for outliers in a non-normally distributed dataset: n: 1,900 Mean: 2,738 StDev: 1,544 Min: 1 Max: 22,102 Anderson-darling: 40 P < 0.005 The boxplot shows the outliers in one ...
0
votes
0answers
5 views

how to find influential data point in an aggregate

In an explanatory study (and not a predictive one), I want to find the influence of an extreme point (from a data set) on an aggregate (may be sum of the all the data points in the set)? I have found ...
2
votes
0answers
28 views

Changing sensitivity (cval) in tsoutliers resulting in unexpected results

I am using the excellent tsoutliers R package to detect outliers (additive outliers, temporary changes etc.), but the cval ...
0
votes
0answers
8 views

Calibrate sensors to each other?

I have 40 temperature sensors that have been in a controlled environment (over 40°C range) they are all factory Calibrated to 0.5°C for room temperature. What would be a good statistical method to ...
0
votes
0answers
7 views

Robuste classification of ground and no-ground points in dense forest LiDAR Airborne

This is my first project on my graduation so take easy. Im studying methods to remove outliers on LiDAR airbone data in the tropical amazon rainforest. Remove outliers above the trees is easy, the ...
1
vote
1answer
25 views

what is the appropriate one-class classifier for sequenced categorical data?

Can someone suggest an algorithm for an outlier detection system? My requirements are: it supposed to be a one class classifier, where on training phase, it only fed 'normal' data however the normal ...
0
votes
0answers
9 views

Question about MCD and R's cov() function

I'm trying to identify outliers using mahalanobis distance acquired by the covariance matrix determined by the MCD-selected observations of a data matrix X (i.e. the vectors contained within the ...
1
vote
1answer
41 views

Comparison of Stahel-Donoho and Minimum Covariance Determinant estimation

I am interested in detecting multivariate outliers in a low dimensional data set ($n<p$). Various high-breakdown robust methods for multivariate settings such as Stahel-Donoho and Minimum ...
0
votes
0answers
13 views

usage of adjbox() in R [duplicate]

Anyone here who could describe the usage of adjbox()? Is it used for very skewed distributions to have a more generous range like the boxplot? What is the formula like mean+1.5*IQR in boxplots? Is it ...
0
votes
0answers
15 views

chisquare outlier test assumptions

what are the assumptions of the chisquare outlier test? ( in R: chisq.out.test()) Is it only applicable if the data follow a certain Distribution? What is the idea of ist Definition of outliers? ...
0
votes
0answers
35 views

Identify outlier usage intervals in time-series data

I want to find outliers in power consumption in real-time, at hourly rate, i.e., at the end of the hour, I should say whether power consumption in current hour was outlier/anomalous or not. Approach: ...
0
votes
0answers
23 views

Appropriate testing for outliers [closed]

I want to perform an outlier-test for my data. The distribution of the data looks like a hyperbola (there are many values near 0, all values are >0). Sample size of data is 5000. I want to find an ...
0
votes
0answers
10 views

Finding outliers on LiDAR for forests

I'm working with LiDAR data in college and focusing on preprocessing. I'm still graduating and this is my first project. So, I used the library of c++, Point Cloud Library (PCL) to deal with the ...
2
votes
0answers
33 views

Dealing with noisy/mislabelled dataset

I have several datasets where each instance has numeric label assigned by a human that can take values between 1 and 5. After doing a manual inspection of one of these datasets, I noticed the ...
0
votes
1answer
30 views

Finding outlier values for non-normally distributed data

I have univariate data (38 is the sample size).The distribution is certainly not normal. How can I find the outliers? I used z-score but am not getting a desired result.
1
vote
0answers
19 views

Transformation and/or Winsorizing?

I want to compare two group of 24 and 28 people with t-test on type of activity (5 different's types of activity and a total), later one the same value will be use in regression logistic. If you ...
0
votes
0answers
41 views

PROC ROBUSTREG or PROC NLMIXED in SAS, to down-weight effects of non-normality and outliers

We are conducting an OLS regression in SAS. We have performed all diagnostic tests and concluded that there is non-normality in the residuals, influential outliers and heteroskedasticity. We would now ...
0
votes
2answers
58 views

How do I remove outliers in dataset?

I have a data-set (185 rows) with 20 predictors and 1 dependent variable. I have applied Cook's distance and then 4/N formula to remove some of the outliers in 1st iteration. Should I do this ...
3
votes
1answer
37 views

Do points with high Cook's distance necessarily have a high standardized residual, and vice-versa?

I have two questions below: Could a data point be an influential point if its cook distance is outstanding(greater than 4/(n-p-1)) while its standardised residual is less than 2? It looks like to me ...
0
votes
0answers
19 views

With or without outlier and/or parametric vs non-paramteric

I'm comparing two group on the number or movie seen in the last mont (30 people in each group). Spss say that there 4 outlier but when I'm lookin with the z score I only find 1. If I do a t-test it'...
1
vote
0answers
25 views

Winsorizing, just the outlier or all the value?

I have an outlier in my data set. I want to use the winsorizing quartile (to change the outlier to the 5th% and/or 95th%). Looking at the quartiles, sometimes I have more values than just my ...
1
vote
1answer
33 views

Outlier transformation for t-test

In a normally distributed sample with one outlier, some suggest changing the value of the outlier to be one unit above the next highest score. If you are doing a comparison test on two groups using ...
1
vote
1answer
40 views

Robust statistic for representing small dataset with outliers and representing them graphically

I'm evaluating different systems varying a certain parameter common to all of them. Let this parameter be x. At each x value I ...
0
votes
0answers
6 views

Using historic water flux data to detect the existence of leakage, where should I start?

This question is also linked to How to detect a relatively small level shift(leakage) in an hourly water flux time series in an area? which I asked a week ago... Background I've got a series of ...
0
votes
0answers
9 views

Replacing an outlier in one of two terms with a mean

Should I replace values with means? I have an experiment and I know (from the literature) that 3000ms reaction times means the participant is not attentive (the outliers reach 20000ms). I'm analyzing ...
1
vote
1answer
47 views

What comes first: outlier detection or model selection?

I'm fitting a GLMM (mixed logistic regression) in R. I have five covariates. For model selection, I'm using glmmLasso() (in R) to determine which of the five covariates and their interactions should ...