An outlier is an observation that appears to be unusual or not well described relative to a simple characterization of a dataset.

learn more… | top users | synonyms (1)

0
votes
0answers
16 views

How to find mobile app usage outliers?

I am doing some analysis on the usage data of my company's mobile app. However, the QA team will do some prod-test at any time, which generates enormous amount of usage data and also obscure the ...
0
votes
0answers
21 views

Dealing with outliers: Clustering

I am working with a dataset in R that I will be doing cluster analysis on and I am trying to determine the best way to deal with the outliers.I have twelve variables and most variables have between ...
1
vote
0answers
12 views

Filtering outliers: maintaining consistency over time with newly added records

Each item in a table has a price and is assigned to a region. I filter out outliers by identifying and deleting the records whose prices are more than 3 standard deviations away from the average for ...
1
vote
1answer
37 views

MAD formula for outlier detection

Does anyone know what is the name of this formula? $$M_i = \displaystyle\frac{0.6745(x_i - \hat{x})}{\mathrm{MAD}}$$ where $\textrm{MAD}$ is the median absolute deviation and $\hat{x}$ is the median ...
2
votes
0answers
19 views

How to call “Inliers” and “Outliers” in French

I asked this on the French Exchange site, but this is stat related so... How do you say "Inliers" and "Outliers" (as with RANSAC) in French? The Wikipedia article doesn't translate them, but honestly ...
3
votes
1answer
97 views

How to find set of directions in Stahel-Donoho outlyingness measure?

Currently I’m trying to understand and use the Stahel-Donoho outlyngness measure. But unfortunately I’ve got a problem in the part where one is taking the maximum over the set of directions. I found ...
0
votes
0answers
12 views

Determine cause of anomaly

I'm trying to perform some kind of clustering based anomaly detection for time series and it gives me solid results. What I am interested in is - are there any methods to determine cause of anomally? ...
0
votes
0answers
8 views

Characterizing “typical behavior” for events?

I need to build a model to characterize what is typical for a series of events, which in turn will be used to flag atypical events. As an example, think of credit card purchases (how often? what ...
1
vote
0answers
47 views

Multivariate outlier detection for PLS model

I am working with a PLS model (library pls) in R, where I am developing calibration models for NIRS data. I have been using other commercial software before that allowed me to detect outliers based on ...
0
votes
0answers
20 views

Fundamental Issues with Influence weighted resampling for bootstrapped predictions

I have a large database 1mill+ from which it is known that there are many influential points and outliers. I am interested in generating a series of predictions from subsets (1,000+) of the data and ...
3
votes
2answers
261 views

Analyze scatter plot

I want to study the relationship between two variables. I've got the following scatter plot. But now I'm hesitating on what to do with this: Should I check the assumptions of OLS and then use the ...
0
votes
0answers
14 views

R: Analyze scatter plot [duplicate]

I want to study the relationship between twe variables. I've got the following scatter plot. But now I'm hesitating on what to do with this: 1) Should I check the assumptions of OLS and then use ...
1
vote
1answer
72 views

What are some useful robust and scalable approaches towards anomaly detection of a time series data?

What are some useful robust and scalable approaches/methods towards anomaly detection of a time series data? I am mainly looking for some practical approaches carried out using Python, R, Java, etc. ...
4
votes
0answers
59 views

How to quickly identify participants responding randomly to self-report psychometric tests with many items?

Many psychological studies involve getting participants to answer a hundred or more closed ended questions. A standard context would be a personality test with 100 items where each item is answered on ...
0
votes
1answer
20 views

unsupervised clustering with “unclassified” items

I have data (some behavioral features, measured on some scales) on people. I want to cluster people based on these features. This is an unsupervised scenario, as I have no prior knowledge on the ...
0
votes
0answers
19 views

How to find a wrong predictor value based on other correlated predictors

I have five correlated predictors, ref the following pairs plot: Now I suspect that sometimes a predictor is wrong, as these come from different sources. In other words, four of the predictors of ...
0
votes
0answers
23 views

Top coding: Replace outliers by the value of the mean +/- 6 times the standard deviation in SAS

I'm wondering if anyone can give me some help with this.. I'd like to perform a cluster analysis and this method is very sensitive to outliers. Therefore, top coding is used in similar research: ...
1
vote
0answers
22 views

how to determine outliers in sample affected by ascertainment bias

I don't know if this is a really silly question as I'm in no way a statistician and I don't know if this is something that's actually quite rudimentary... Thanks for reading in advance too it got kind ...
8
votes
1answer
225 views

Can we use leave one out mean and standard deviation to reveal the outliers?

Suppose I have normally distributed data. For each element of the data I want to check how many SDs it is away from the mean. There might be an outlier in the data (likely only one, but might be also ...
0
votes
0answers
48 views

Identifying multivariate outliers in a large sample with missing data, using SPSS

I'm a psychology PhD student doing analysis on a relatively large set of data, obtained via online surveys. The purpose of the study is largely to determine normative data for a population of adults, ...
4
votes
1answer
67 views

Outlier Detection in Time-Series: How to reduce false positives?

I'm trying to automate outlier detection in time-series and I used a modification of the solution proposed by Rob Hyndman here. Say, I measure daily visits to a website from various countries. For ...
0
votes
0answers
12 views

What effect, if any, do outliers have on mediation analysis with bootstrapping?

I am running a mediation analysis spread over 6 models. Analysis is performed using the PROCESS macro. Each model includes 1 IV, 2 parallel mediators, and 1 DV. In a couple of the IVs, a number of ...
-1
votes
1answer
50 views

Best clustering technique for outlier detection?

I have around 15-20 points every second, and I would like to detect outliers based on -their density along x-axis , that means if I am using k-mean clustering then I specify that in x-direction max ...
0
votes
0answers
18 views

Kalman filter before or after outlier removal?

I am getting radar data points in form of (x,y) coordinate system relative to my position every ms.[around 10-15 data points]. Now, inorder to have better position estimate of the points, I would like ...
2
votes
0answers
37 views

Use cases for a unique metric

So I was thinking about this question: Maximum minus average? This question is on hold while work calls me. When I get a chance I am going to substantially update it. Background: At my previous ...
0
votes
1answer
31 views

Correlation and Outliers

I want to know if someone has some experience working the 'issue' im encountering. I have a series of arrays, 18, time series on a 14 year period, I want to build a correlation matrix with these ...
1
vote
1answer
47 views

Clustering based anomaly detection

I'm trying to implement anomaly detection based on clustering. I'm hopping for confirmation of my approach, and I'm exposing my idea, being aware that I could have miss something in my analysis, so ...
0
votes
2answers
134 views

how to detect outliers from residual plot?

I have the following residual plot. Can I detect outliers from residual plot? I want to remove 200 outliers in my data set, but I do not know how should I do that in R ? residual plots: scatter ...
0
votes
0answers
86 views

Estimating the uncertainty of a bias and a scatter

I have one single set of observational data. Assuming I know the right answer for one property of this data set and then I use one tool to measure this quantity. To get an estimate of the amount of ...
1
vote
0answers
16 views

Factor analysis using “outliers-only” time series

Some background I run a factor analysis of a time series $Y$ using a standard OLS model with n+1 independent variables $(F,X_1...X_n)$, where $F$ is the main factor (from an explanatory power ...
1
vote
0answers
17 views

Sample datasets with known outliers for IQR, Q-test and Z-test [closed]

Is anyone aware of a source for sample data sets with known outliers? I've been looking around for years but haven't come up with a solution, short of creating my own limited database. Sets with ...
0
votes
0answers
13 views

Outlier detection in weighted time series

Given a set of observations $X _n = \{x_1, \dots, x_n\}$, and a new observation $x_{n+1}$, we aim to find whether $x_{n+1}$ is outlier. One approach to solve this problem is to check whether ...
0
votes
0answers
24 views

Intuition on One Class Support Vector Machines

I am having trouble understanding how one-class SVMs work. They were introduced in a paper by Scholkopf and others (and can be found here). One-class SVMs perform "novelty detection", where a point ...
1
vote
3answers
63 views

ANOVA with outlier group

This is my first question (previously the search function has been enough), so please bear with me. I have a very simple experimental design with one outcome variable and 5 groups. My typical ...
0
votes
0answers
22 views

Any outliers test for any distribution?

I have different data series of rainfall values (mm), and some days there are points out of the normal trend. When I apply multiple distributions to try to fit the series they are different ...
1
vote
0answers
21 views

Handling outliers in the target variable

I'm using a support vector regression model. I know the target variable has some outliers and modeling the data directly leads to bad results (Rsquare close to 0.2). I'm pretty sure the outliers are ...
0
votes
0answers
15 views

The outlier in non-random data

I have tested a time series data by using Run Test. The result shows that it is non random. How could I detect which data point(s) caused that non-randomness? Is there have any methods I can use? ...
1
vote
2answers
65 views

One Class SVM strange decision boundary

I am trying to plot the decision boundary of a One Class SVM. This is a 2 dimensional representation of my training data And here the picture of the prediction obtained on the training data ...
0
votes
0answers
51 views

How to represent outliers for multi dimensional data (local outlier factor)

Below graph taken from http://en.wikipedia.org/wiki/Local_outlier_factor displays "LOF scores : LOF image : This is great for two dimensional data but what about data > than two dimensions. How ...
0
votes
0answers
10 views

Mislabeled training instance detection and relabeling

I have some text data represented by sparse BOWs features ( ~ 5k features). This data must be classified into (~20) categories, however my training labels data appear to be very noisy (> 20 % wrong ...
0
votes
0answers
52 views

Time-series detection algorithm for multi-seasonal data using Python

My data: I have two seasonal patterns in my hourly data... daily and weekly. For example... each day in my dataset has roughly the same shape based on hour of the day. However, certain days like ...
0
votes
0answers
15 views

How can I reach the plot of outlier effetc of a TC outlier?

In this article there is a plot of the outlier effect. Detecting Outliers in Time Series (LS/AO/TC) using tsoutliers package in R. How to represent outliers in equation format? How can I have this ...
2
votes
3answers
110 views

What is “the shortest half of the data”?

Here is a histogram (realized with JMP) displaying two types of box plot called outlier box plot and quantile box plot. Right below, there are a bunch of explanations of the meaning of the ...
0
votes
0answers
51 views

Finding transformation function for a distribution that looks like exponential

Suppose that we have two data sets, R and P. R is larger than or equal to ...
3
votes
2answers
238 views

Outliers and the mean

I would like to know what the following example is called in mathematics: In a gymnastics competition the judges scored a competitor as 10, 8, 3, 7, 7, 9, and 8. I recall that the ending score was ...
1
vote
0answers
70 views

Transform long tail data set to bell curve [closed]

I'm analyzing a data set that has an extremely long tail, and I'm looking for a way to transfer the data into a bell curve so I can apply statistical analysis to it. Hope this makes sense, any help ...
3
votes
1answer
42 views

Identifying outliers for within-patient numerical data

I have a simple dataset of people's heights, many people with measurements on multiple days (once a year for 10 years, say). I have the date of each measurement. Some of the height values are ...
0
votes
0answers
42 views

Bootstrapping in SAS - PROC LOGISTIC - Next steps ? how to score / perform diagnostics?

My question is as follows. I am referencing the following paper by David Cassell - wherein David talks about bootstrapping techniques in SAS using PROC SURVEYSELECT (many thanks to David - truly a ...
12
votes
4answers
765 views

Good form to remove outliers?

I'm working on statistics for software builds. I have data for each build on pass/fail and elapsed time and we generate ~200 of these/week. The success rate is easy to aggregate, I can say that 45% ...
0
votes
1answer
82 views

Outlier detection using k-means in a binary classification problem

I'm using k-means in every class of a binary classification problem and remove samples that have high distance from center of my features (21 features so 21 ...