An outlier is an observation that appears to be unusual or not well described relative to a simple characterization of a dataset.

learn more… | top users | synonyms (1)

1
vote
1answer
12 views

Small number of points driving correlation

I am currently looking at trying to find high correlations in a series of data. I initially just thought I could check the correlation of the data and choose those with the highest correlation but ...
1
vote
0answers
9 views

How to do multivariate outlier detection in mixed data with category?

I have a data table where the entries are in the following format. The first column is category, which represent the product category. I have 5 such categories. ...
1
vote
0answers
22 views

Iterative process for removing extreme samples

My samples follow heavy tail distributions. I use a process to detect and remove "extreme" samples that goes like this: Measure mean and standard deviation of samples. Remove samples higher than ...
2
votes
1answer
130 views

Detect outliers in very small data set

I have a data set that includes the different response times of a user that is visiting a web application. For example, a visitor enters www.test.com in the browser and navigates through this domain ...
-2
votes
0answers
50 views

Method to identify samples lying outside the normal distribution

In my previous questions I was looking for a method identify samples that had a variability significantly higher than the rest of my dataset. Methods to determine reliability of measurements using ...
0
votes
0answers
22 views

Anomaly detection for one feature vector

I have a $n$-dimensional vector of ordered multiple testing $p$-values and I would like to reject the first values that are under a certain threshold $\alpha$. I am looking at this problem as an ...
4
votes
1answer
144 views

Detecting Outliers in Time Series (LS/AO/TC) using tsoutliers package in R. How to represent outliers in equation format?

Comments: Firstly I would like to say a big thank you to the author of the new tsoutliers package which implements Chen and Liu's time series outlier detection which was published in the Journal of ...
2
votes
1answer
35 views

Find rows in data that are statistically different from the mean

I have the following data for various event locations. Each event can either be a success or failure (binary values). Thus the mean = percentage of successes. The data represent the history of events ...
2
votes
2answers
64 views

Outlier problems

Having built a regression model with an ordinal response variable and predictors comprised of categorical and continuous nature, I have some questions that pertain to one of the final goals, i.e. ...
0
votes
1answer
30 views

Identifying subsets for outlier detection in local outlier factor

I am trying to gain better understanding of the idea of local outliers (as discussed in this pdf) and how the function is implemented. Here are the key passages from the pdf: Local outliers: ...
4
votes
2answers
82 views

Outlier detection using regression

Can regression be used for out lier detection. I understand that there are ways to improve a regression model by removing the outliers. But the primary aim here is not to fit a regression model but ...
6
votes
1answer
153 views

Robust estimation of kurtosis?

I am using the usual estimator for kurtosis, $\hat{K}=\frac{\hat{\mu}_4}{\hat{\sigma}^4}$, but I notice that even small 'outliers' in my empirical distribution, i.e. small peaks far from the center, ...
1
vote
0answers
15 views

What's the proper way to do automatic iterative outlier rejection?

I know this is a touchy subject, so I'll proceed with caution... I am building regression models for some hundreds of thousands of data files containing only two columns. The sensor collecting these ...
0
votes
1answer
36 views

Linear Regression with Outlier accounting in Bugs

I'm trying to redo an exercise in BUGS from this webpage: a linear regression over a data set with some outliers, using a model that accounts for them. This model uses a mixture of signal and noise ...
0
votes
1answer
76 views

Outlier detection using clustering and dissimilarity matrix in R

I have some problems in finding the outliers using clustering. The data.frame is ~20000 observations and each row has mixed types of variables(numeric, nominal and binary). What I want to do is to ...
2
votes
0answers
41 views

How to detect outliers with longitudinal data?

I am running a pooled OLS and Random Effects (RE) model and I would like to test for whether there are any outliers. I know how to do this for OLS, but I just dont know how to do it for Random ...
2
votes
2answers
37 views

How to appropriately represent certain outliers

I am working with a dataset representing a material's 'Range of Coverage', or, a calculated amount of time it is expected to stay in stock. This calculation is based on a material's usage during a ...
11
votes
2answers
104 views

using neighbor information in imputing data or find off-data (in R)

I have dataset with assumption that nearest neighbors are best predictors. Just a perfect example of two-way gradient visualized- Suppose we have case where few values are missing, we can easily ...
0
votes
0answers
54 views

Outlier detection with ROBPCA for multivariate poisson/non-normal data

It is stated here[1] that we can use ROBPCA to detect outliers for multivariate data. After reading the manual ([2] page 12 : "multivariate normal model etc."), I think the ROBPCA method is also ...
0
votes
0answers
29 views

Outliers in a bimodal distribution

Somewhat related to this post. However my question is a little specific. I have some data from gene expression quantification from single cells using RNA sequencing. I haven't had a look at it but I ...
2
votes
0answers
20 views

Von Mises distribution to detect outliers

I am working out the difference between two angles from a circle, and I work out the mean difference across 96 trials in 10 separate samples. In order to detect outliers for statistical analysis, ...
3
votes
3answers
165 views

Anomaly detection: what algorithm to use?

Context: I'm developing a system that analyzes clinical data to filter out implausible data that might be typos. What I did so far: To quantify the plausibility, my attempt so far was to normalize ...
2
votes
0answers
43 views

Can Chauvenet's criterion be used with non-normal data?

Can I use Chauvenet's criterion on set of observations where a normal distribution cannot be assumed?
4
votes
0answers
41 views

Outlier detection in beta distributions

Say I have a large sample of values in $[0,1]$. I would like to estimate the underlying $\text{Beta}(\alpha, \beta)$ distribution. The majority of the samples come from this assumed ...
13
votes
3answers
661 views

How to assess skewness from a boxplot?

How to decide skewness by looking at a boxplot built from this data: 340, 300, 520, 340, 320, 290, 260, 330 One book says, "If the lower quartile is farther from the median than the upper quartile, ...
1
vote
1answer
38 views

Recognize outliers in a set as data collected

Suppose I am aggregating data at multiple granularities, where each key is associated with the number of counts collected during the time interval. EX: 1 minute { x:2, y:5, z:3, a:312 } This is a ...
3
votes
1answer
60 views

How to interpet residual plot?

As a part of a design of experiments course I'm taking, I ran an experiment at home. The experiment was checking how water boiling time changes under certain factors (5 overall factors) all which had ...
0
votes
0answers
20 views

Can splines produce a plot with more than one fixed effect? If so, how can I correct for a fixed effect?

I have my statistician supervisor wanting me to plot a series of graphs using splines. On the x axis I have volume of milk, on the y axis days. Now, he says I need to "correct for age and birth ...
0
votes
0answers
26 views

standard deviation set for spline cubic regression model

I'm attaching two sets I ran in SAS, regarding spline cubic regression models. I'm new to stats and trying my best to learn but I'm stuck here. How do I get SAS to select outliers based on two or 3 ...
1
vote
0answers
46 views

When to remove/replace outliers for different types of analysis

I have a general question about handling outliers when doing univariate and multivariate analysis, but I'm going to present my specific situation for discussion clarity. I'm dealing with ecological ...
2
votes
1answer
33 views

What is an indicator in statistics and why is it used to refine models?

In a problem we are asked to "refine the fitted model by using an indicator for the outlier". What does it mean to use an indicator for an outlier?
0
votes
0answers
55 views

Detecting outliers in non parametric data

what is the best (and easy) way to detect outliers in spss for skewed, not exactly normally distributed data? Thanks a lot
0
votes
1answer
31 views

Outliers and Linearity for EFA

I am having 56 likert scaled items for IV and 28 likert scaled items for dv. As to fulfill the assumption for EFA, outliers and linearity need to be checked. Can anyone tells me what method/analysis ...
0
votes
0answers
21 views

Discriminant analyses with NIRS in R

I am working with a NIR matrix consistent of 134 rows (samples) and 1529 columns (wavelengths), from which I want to discriminate between two categories (species). I have successfully used the plsda ...
1
vote
0answers
55 views

Remove outliers and calculate average

I am trying to understand (by reverse engineer) the method a program is using to calculate an average value over a population. The program displays a (very very large) image consisting of tiles. Each ...
3
votes
3answers
151 views

What problems do non-normality in predictor variables cause for a multiple regression analysis?

I am talking about a situation in which I have several continuous predictor variables predicting a continuous outcome. One of the predictors has a very non-normal distribution and has some wild ...
1
vote
0answers
86 views

What is the code for cubic spline regression model in SAS

I ran an experiment that identified lame and non-lame cows every day for 325 days from a pool of 936 cows in one herd. At the same time, I collected data on various variables like milk volume, fat and ...
1
vote
0answers
27 views

Detecting heterogeneity in groups

This is a mockup of a dataset I am currently working on: ...
0
votes
0answers
59 views

Outlier removal and standardization of variables

In a multifactor model of stock returns, I am considering several variable $X_1$, $X_2$, ... , $X_n$ as explanatory variable. However, before including the variables in the model, I would like to: ...
0
votes
1answer
64 views

Outliers for Normalization: Is it important?

I'm new to statistics and have to do some research analysis using SPSS. I need to know, what is the purpose of removing outliers? I want to normalize my set of data obtained through a national survey, ...
1
vote
0answers
63 views

Outlier detection in binary classification

I have a question about outlier detection in my system. I’m designing a system (in Matlab) that optimize both features and parameters of a classification method (like mlp) together with optimization ...
0
votes
0answers
15 views

mvoutlier vs influence measures

I am exploring the mvoutlier package and comparing it against conventional influence measures such as Cook's Distance, Leverage, DFFITS etc. I have not been able to get my hands around comparative ...
0
votes
0answers
9 views

Significant difference real or due to the internal variability?

In my data I have 9 different sets of data for 2 different groups. Each one of these datasets is the same measurement changing over the time. If I make a graph, I can see 9 lines for each group. I did ...
2
votes
1answer
39 views

Standard deviations and significance values

I was wondering if data 2 SD from the mean is deleted as outliers, is it possible there after to report 0.01 significance values? Thanks!
0
votes
1answer
57 views

Filter out “abnormally large” values from a list of data

For example, the lists can be something like: $$\{1.123213, 5.154543, 2.134121, 7.34534, 12.223432, 8.16571, 100.45645, 222.423\}$$ I want to remove $\{100.45645, 222.423\}$. $$\{232.123213, ...
1
vote
0answers
51 views

Term used to describe a particular sample

Suppose that we know that captured data follow a particular distribution. We have 10 sample. 9 of them are close to the supposed distribution but the last sample seems to be very distant, or event ...
1
vote
1answer
81 views

Sample-adjusted meta-analytic deviancy macro/syntax for Excel, SPSS, MPLUS

I need to compute the sample-adjusted meta-analytic deviancy (SAMD) as part of an outlier search with meta-analytic data. Because we have a very large amount of meta-analytic data, computing this ...
0
votes
1answer
44 views

least trimmed squared for regression

i'm new in statistics. hope you can help me on the following: i want to use least trimmed squared (LTS) for regression. below is the coding in R: ...
1
vote
0answers
42 views

qqPlot with Two Outliers

If I have a QQ plot with two extreme outliers (picture below) how should I interpret it? Do I drop the outliers? Can I treat it as normal?
0
votes
0answers
27 views

Cross-estimating the independent variables to exclude outliers

Purpose: pragmatic data mining and prediction, NOT for publication or science Data: observations from nature, so a high degree of stability is expected in the relationships N: approx. 15 k I am ...