An outlier is an observation that appears to be unusual or not well described relative to a simple characterization of a dataset.

learn more… | top users | synonyms (1)

1
vote
0answers
17 views

how to determine outliers in sample affected by ascertainment bias

I don't know if this is a really silly question as I'm in no way a statistician and I don't know if this is something that's actually quite rudimentary... Thanks for reading in advance too it got kind ...
6
votes
1answer
111 views
+50

Can we use leave one out mean and standard deviation to reveal the outliers?

Suppose I have normally distributed data. For each element of the data I want to check how many SDs it is away from the mean. There might be an outlier in the data (likely only one, but might be also ...
0
votes
0answers
34 views

Identifying multivariate outliers in a large sample with missing data, using SPSS

I'm a psychology PhD student doing analysis on a relatively large set of data, obtained via online surveys. The purpose of the study is largely to determine normative data for a population of adults, ...
0
votes
0answers
19 views

Incorporating outlier points into a unified prediction model [closed]

There is a data set including multiple outlier points. If we want to build a regression model based on these points, what are the general approach to handle these outlier points? Are there any good ...
2
votes
1answer
49 views

Outlier Detection in Time-Series: How to reduce false positives?

I'm trying to automate outlier detection in time-series and I used a modification of the solution proposed by Rob Hyndman here. Say, I measure daily visits to a website from various countries. For ...
0
votes
0answers
10 views

What effect, if any, do outliers have on mediation analysis with bootstrapping?

I am running a mediation analysis spread over 6 models. Analysis is performed using the PROCESS macro. Each model includes 1 IV, 2 parallel mediators, and 1 DV. In a couple of the IVs, a number of ...
-1
votes
1answer
39 views

Best clustering technique for outlier detection?

I have around 15-20 points every second, and I would like to detect outliers based on -their density along x-axis , that means if I am using k-mean clustering then I specify that in x-direction max ...
0
votes
0answers
11 views

Kalman filter before or after outlier removal?

I am getting radar data points in form of (x,y) coordinate system relative to my position every ms.[around 10-15 data points]. Now, inorder to have better position estimate of the points, I would like ...
2
votes
0answers
35 views

Use cases for a unique metric

So I was thinking about this question: Maximum minus average? This question is on hold while work calls me. When I get a chance I am going to substantially update it. Background: At my previous ...
0
votes
1answer
28 views

Correlation and Outliers

I want to know if someone has some experience working the 'issue' im encountering. I have a series of arrays, 18, time series on a 14 year period, I want to build a correlation matrix with these ...
1
vote
1answer
36 views

Clustering based anomaly detection

I'm trying to implement anomaly detection based on clustering. I'm hopping for confirmation of my approach, and I'm exposing my idea, being aware that I could have miss something in my analysis, so ...
0
votes
2answers
85 views

how to detect outliers from residual plot?

I have the following residual plot. Can I detect outliers from residual plot? I want to remove 200 outliers in my data set, but I do not know how should I do that in R ? residual plots: scatter ...
0
votes
0answers
77 views

Estimating the uncertainty of a bias and a scatter

I have one single set of observational data. Assuming I know the right answer for one property of this data set and then I use one tool to measure this quantity. To get an estimate of the amount of ...
1
vote
0answers
16 views

Factor analysis using “outliers-only” time series

Some background I run a factor analysis of a time series $Y$ using a standard OLS model with n+1 independent variables $(F,X_1...X_n)$, where $F$ is the main factor (from an explanatory power ...
1
vote
0answers
16 views

Sample datasets with known outliers for IQR, Q-test and Z-test [closed]

Is anyone aware of a source for sample data sets with known outliers? I've been looking around for years but haven't come up with a solution, short of creating my own limited database. Sets with ...
0
votes
0answers
11 views

Outlier detection in weighted time series

Given a set of observations $X _n = \{x_1, \dots, x_n\}$, and a new observation $x_{n+1}$, we aim to find whether $x_{n+1}$ is outlier. One approach to solve this problem is to check whether ...
0
votes
0answers
21 views

Intuition on One Class Support Vector Machines

I am having trouble understanding how one-class SVMs work. They were introduced in a paper by Scholkopf and others (and can be found here). One-class SVMs perform "novelty detection", where a point ...
1
vote
3answers
55 views

ANOVA with outlier group

This is my first question (previously the search function has been enough), so please bear with me. I have a very simple experimental design with one outcome variable and 5 groups. My typical ...
0
votes
0answers
19 views

Any outliers test for any distribution?

I have different data series of rainfall values (mm), and some days there are points out of the normal trend. When I apply multiple distributions to try to fit the series they are different ...
1
vote
0answers
19 views

Handling outliers in the target variable

I'm using a support vector regression model. I know the target variable has some outliers and modeling the data directly leads to bad results (Rsquare close to 0.2). I'm pretty sure the outliers are ...
0
votes
0answers
15 views

The outlier in non-random data

I have tested a time series data by using Run Test. The result shows that it is non random. How could I detect which data point(s) caused that non-randomness? Is there have any methods I can use? ...
1
vote
2answers
57 views

One Class SVM strange decision boundary

I am trying to plot the decision boundary of a One Class SVM. This is a 2 dimensional representation of my training data And here the picture of the prediction obtained on the training data ...
0
votes
0answers
43 views

How to represent outliers for multi dimensional data (local outlier factor)

Below graph taken from http://en.wikipedia.org/wiki/Local_outlier_factor displays "LOF scores : LOF image : This is great for two dimensional data but what about data > than two dimensions. How ...
0
votes
0answers
8 views

Mislabeled training instance detection and relabeling

I have some text data represented by sparse BOWs features ( ~ 5k features). This data must be classified into (~20) categories, however my training labels data appear to be very noisy (> 20 % wrong ...
0
votes
0answers
35 views

Time-series detection algorithm for multi-seasonal data using Python

My data: I have two seasonal patterns in my hourly data... daily and weekly. For example... each day in my dataset has roughly the same shape based on hour of the day. However, certain days like ...
0
votes
0answers
15 views

How can I reach the plot of outlier effetc of a TC outlier?

In this article there is a plot of the outlier effect. Detecting Outliers in Time Series (LS/AO/TC) using tsoutliers package in R. How to represent outliers in equation format? How can I have this ...
2
votes
3answers
93 views

What is “the shortest half of the data”?

Here is a histogram (realized with JMP) displaying two types of box plot called outlier box plot and quantile box plot. Right below, there are a bunch of explanations of the meaning of the ...
0
votes
0answers
40 views

Finding transformation function for a distribution that looks like exponential

Suppose that we have two data sets, R and P. R is larger than or equal to ...
3
votes
2answers
232 views

Outliers and the mean

I would like to know what the following example is called in mathematics: In a gymnastics competition the judges scored a competitor as 10, 8, 3, 7, 7, 9, and 8. I recall that the ending score was ...
1
vote
0answers
56 views

Transform long tail data set to bell curve [closed]

I'm analyzing a data set that has an extremely long tail, and I'm looking for a way to transfer the data into a bell curve so I can apply statistical analysis to it. Hope this makes sense, any help ...
3
votes
1answer
38 views

Identifying outliers for within-patient numerical data

I have a simple dataset of people's heights, many people with measurements on multiple days (once a year for 10 years, say). I have the date of each measurement. Some of the height values are ...
0
votes
0answers
28 views

Bootstrapping in SAS - PROC LOGISTIC - Next steps ? how to score / perform diagnostics?

My question is as follows. I am referencing the following paper by David Cassell - wherein David talks about bootstrapping techniques in SAS using PROC SURVEYSELECT (many thanks to David - truly a ...
12
votes
4answers
755 views

Good form to remove outliers?

I'm working on statistics for software builds. I have data for each build on pass/fail and elapsed time and we generate ~200 of these/week. The success rate is easy to aggregate, I can say that 45% ...
0
votes
1answer
52 views

Outlier detection using k-means in a binary classification problem

I'm using k-means in every class of a binary classification problem and remove samples that have high distance from center of my features (21 features so 21 ...
1
vote
1answer
30 views

Leverage - influential points

Do we look at the absolute value of the leverage or the relative value? For instance, based on the chart below, the largest leverage is about 0.023, it is big compared to other data points, but I'm ...
1
vote
0answers
30 views

Order of preprocessing steps in a binary classification problem

I have these stages (ordered) for preprocessing in my binary classification problem. Dividing data based on criteria (class1 and class2 databases) Outlier ...
1
vote
0answers
57 views

Outlier Removal in Non-Normal Data

I am currently working on a project in which I have to eliminate outliers from non-normally distributed data sets. The data sets are subsets of a fairly large database (order of millions of ...
0
votes
1answer
32 views

Is my understanding of how to calculate the reachability distance in local outlier factor correct?

Reading lof implementation at : http://www.cse.ust.hk/~leichen/courses/msc-it5210/lectures/LOF_Example.pdf the local reachability distance is given as : I don't fully understand this equation as ...
2
votes
1answer
21 views

Is Outlier detection in two separate databases is equal to one combined database?

Suppose that we have two databases : Database_1 and Database_2 . Database_1 has 300 samples ...
1
vote
0answers
23 views

How to pick the “sigma” when sigma clipping?

In astronomy (and I'm sure in other places as well) it is very common to use sigma-clipping as the outlier rejection scheme. The idea is that you regress the data, subtract the fit from the points, ...
0
votes
1answer
22 views

Use random forest outliers to detect group of variables

I have a input data and an output binary variable . The y value is 1 if the patient get ill. ...
0
votes
0answers
38 views

Publication bias - trim and fill - macro/syntax

I'm looking for an SPSS or Excel macro/syntax that will allow me to compute Duval and Tweedie’s trim and fill procedure (see paper here: ...
1
vote
1answer
110 views

Outlier detection in out-sample data for the purpose of classification

This is the most widely used method for outlier detection in econometrics and statistical problems. X is our data that we're searching for outliers in it (in ...
0
votes
2answers
69 views

Can decision tree be used for fraud detection in this way?

A large dataset with more than 100 variables including a target variable. A small portion of target = 1 cases are fraud or due to other errors. I want to identify these target = 1 cases, i.e. fraud or ...
1
vote
0answers
28 views

High dimensional model estimation with outliers

I have a set H of k m-dimensional hyperplanes in n dimensional space, where ...
5
votes
2answers
96 views

Detecting outliers in circular data?

I've a day of the year data about number of event occurence in different sites: A day of the year is circular data. I know that a usual detecting of outliers, for example by boxplot is no use here: ...
1
vote
1answer
44 views

Analyzing regression results

I have done a regression model where i determine the number of cubes (independent variable) based on the amount of units i started with for each product type (dependent variables, ...
1
vote
1answer
25 views

Have conducted a bivariate correlation - want to identify the outlier - is there a 'statistical' way to do this?

I run correlation (on SPSS) between a governance indicator 1996 and the same governance indicator in 2012. As would be expected there is a near perfect correlation. What I am interested in, however, ...
2
votes
1answer
16 views

Small number of points driving correlation

I am currently looking at trying to find high correlations in a series of data. I initially just thought I could check the correlation of the data and choose those with the highest correlation but ...
1
vote
0answers
19 views

How to do multivariate outlier detection in mixed data with category?

I have a data table where the entries are in the following format. The first column is category, which represent the product category. I have 5 such categories. ...