An outlier is an observation that appears to be unusual or not well described relative to a simple characterization of a dataset.

learn more… | top users | synonyms (1)

0
votes
0answers
20 views

Outlier removal and standardization of variables

In a multifactor model of stock returns, I am considering several variable $X_1$, $X_2$, ... , $X_n$ as explanatory variable. However, before including the variables in the model, I would like to: ...
0
votes
1answer
41 views

Outliers for Normalization: Is it important?

I'm new to statistics and have to do some research analysis using SPSS. I need to know, what is the purpose of removing outliers? I want to normalize my set of data obtained through a national survey, ...
1
vote
0answers
23 views

Outlier detection in binary classification

I have a question about outlier detection in my system. I’m designing a system (in Matlab) that optimize both features and parameters of a classification method (like mlp) together with optimization ...
0
votes
0answers
7 views

mvoutlier vs influence measures

I am exploring the mvoutlier package and comparing it against conventional influence measures such as Cook's Distance, Leverage, DFFITS etc. I have not been able to get my hands around comparative ...
0
votes
0answers
8 views

Significant difference real or due to the internal variability?

In my data I have 9 different sets of data for 2 different groups. Each one of these datasets is the same measurement changing over the time. If I make a graph, I can see 9 lines for each group. I did ...
2
votes
1answer
29 views

Standard deviations and significance values

I was wondering if data 2 SD from the mean is deleted as outliers, is it possible there after to report 0.01 significance values? Thanks!
0
votes
1answer
38 views

Filter out “abnormally large” values from a list of data

For example, the lists can be something like: $$\{1.123213, 5.154543, 2.134121, 7.34534, 12.223432, 8.16571, 100.45645, 222.423\}$$ I want to remove $\{100.45645, 222.423\}$. $$\{232.123213, ...
0
votes
0answers
15 views

Detecting influence of one point in non-linear regression (drc package) [migrated]

I fitted an asymptotic regression model to my data using the drc package in R. the model was specified like this: model<-drm(y~x,data=mydata,fct=AR.2()) now, ...
1
vote
0answers
40 views

Term used to describe a particular sample

Suppose that we know that captured data follow a particular distribution. We have 10 sample. 9 of them are close to the supposed distribution but the last sample seems to be very distant, or event ...
1
vote
1answer
42 views

Sample-adjusted meta-analytic deviancy macro/syntax for Excel, SPSS, MPLUS

I need to compute the sample-adjusted meta-analytic deviancy (SAMD) as part of an outlier search with meta-analytic data. Because we have a very large amount of meta-analytic data, computing this ...
0
votes
1answer
37 views

least trimmed squared for regression

i'm new in statistics. hope you can help me on the following: i want to use least trimmed squared (LTS) for regression. below is the coding in R: ...
1
vote
0answers
33 views

qqPlot with Two Outliers

If I have a QQ plot with two extreme outliers (picture below) how should I interpret it? Do I drop the outliers? Can I treat it as normal?
0
votes
0answers
22 views

Cross-estimating the independent variables to exclude outliers

Purpose: pragmatic data mining and prediction, NOT for publication or science Data: observations from nature, so a high degree of stability is expected in the relationships N: approx. 15 k I am ...
0
votes
0answers
25 views

beanplots: plus sign?

I would like to draw bean plots using the statsmodel package for Python. In the example provided on the documentation, I see a red plus sign in each beanplot: What does it represent?
1
vote
0answers
37 views

Removal of outliers

I work in a chemical laboratory and endeavour to estimate Measurement Uncertainty (MU) from sample retests. The procedure is well established. But what I an dealing with is a set of data where some ...
0
votes
2answers
68 views

Determining more than one outlier from a data set

I have a data set of repeated observations and I am trying to determine if any of the observations are outliers. The research I've done has only shown methods that would determine if one value ...
0
votes
0answers
33 views

Cook's distance cut-off value

I have been reading on cook's distance to identify outliers which have high influence on my regression. In Cook's original study he says that a cut-off rate of 1 should be comparable to identify ...
0
votes
0answers
52 views

Using Normal Q-Q Plot to detect outliers in non-normally distributed values

I have values that are not normally distributed since most (132) of the (140) values are 0 or 1. There are a few that are <10, one about 80 and one about 650. Is it valid to use a normal Q-Q plot ...
4
votes
1answer
184 views

In Spearman's rank correlation: 4 out of 26 observations totally change the conclusion, what should I do?

After I remove these 4 observations, the results change from weak and not statistically significant to the opposite (rho=0.559 with p<0.01). Should I just remove them or what should I do? Although ...
3
votes
3answers
132 views

95% confidence interval for a given data set

I have a data set $\mathcal{S}$, which is univariate. I would like to discover the data points which can be thought of as outliers of $\mathcal{S}$. I use $\bar{\mathcal{S}}+2*\sigma(\mathcal{S})$ ...
0
votes
0answers
137 views

Mutual information with outliers

I am using Mutual Information on a data set with some outliers. The univariate variables don't have a known distribution, so I cannot remove them automatically with any neat method, like Grubbs or ...
1
vote
0answers
48 views

Scale independent forecast error metric that works with changing signs

I am trying to analyze a quite large (~25,000 rows) dataset of cash flow forecasts. Receipts and expenses are aggregated, thus I may end up with the following data: ...
2
votes
0answers
87 views

Outliers causing Heywood case in CFA using MLM estimator in lavaan

I am trying to perform a CFA on data (10 indicators: $n=300$) that is severely non-normal but continuous (counts of a clinical behavior over a period of weeks): many cases are at zero, a fair few ...
2
votes
0answers
70 views

Residual analysis and diagnostics for GEE Models in R

Some colleagues asked me to perform a residual analysis on both linear models and generalized estimating equation (GEE) models. I know it is a faux-pas in some circles to remove outliers, but in our ...
1
vote
1answer
58 views

Alternatives to MAD to find a yardstick to assess data

In a paper by Rousseeuw and Croux from 1993 ("Alternatives to the Median Absolute deviation", page 1274, link to pdf), I came across an indicator I'm considering using. The formula is: $$ Sn = C\, ...
3
votes
0answers
47 views

Can I use weights generated by robust regression in a quasipoisson glm in R?

I have response variable count data that should be treated as quasipoisson or something similar. This data also contains outliers which are important to the dataset. I cannot find an r package that ...
0
votes
0answers
30 views

What is the most robust outlier detection and removal technique, particularly for classification datasets?

I'm trying to prepare my datasets for a classification algorithm that I'm testing. It's a tumour classification dataset to be precise, but I'd like to know of techniques that are dataset as well as ...
1
vote
1answer
75 views

Problem with identifying outliers

I'm running a biological experiment with rodents, have two groups (each consists of 26 animals), where one is treated with a chemical, and one is control (saline). In one variable, there doesn't seem ...
5
votes
3answers
231 views

Understanding the confidence band from a polynomial regression

I'm trying to understand the result I see in my graph below. Usually, I tend to use Excel and get a linear-regression line but in the case below I'm using R and I get a polynomial regression with the ...
1
vote
0answers
60 views

A dataset with some very, very poor X variables - distribution and outliers

I have been handed a dataset with many poor variables and what appear to be outliers. Looking at histograms and box plots of the distributions, many cannot even form a box plot due to the ...
1
vote
1answer
64 views

Proper ordering of outlier removal, standardization, downsampling, etc. in a pre-processing pipeline

I have been using several techniques, sometimes in conjunction with one another, in a pre-processing pipeline prior to using the data for supervised machine learning. I was wondering about the ...
0
votes
1answer
113 views

Outlier detection in ARIMA model with R

After fitting my time series with an ARIMA model, I want to test outliers in the residuals' series. Are there any functions in R that could do this test and furtherly test whether the outlier is ...
2
votes
1answer
77 views

Correlation between approximate classification and standard classification

Let's say I have a large collection of emails and I want to classify in two different class: Non-Spam, Spam. Assume this classification process is very costly, but I have access to an approximate ...
0
votes
1answer
31 views

Multivariant, multi class outlier detection of single points

The problem: I have a set of means and covariance matrices of a number of multivariant normal distributions, each having a class label. Then I get single data points, one after the other, to which I ...
0
votes
1answer
62 views

Finding outliers on simple dataset?

I'm pretty inexperienced in statistics and am wondering the best way to tackle a couple of simple questions. I am given the following data on 1,000 hypothetical slot machines: Attempts made, wins, ...
3
votes
0answers
82 views

Publications discussing “checking to see if outliers affect the result”

I have noticed it is somewhat common for people to collect data, perform a statistical test (e.g., t-test) and then also look for possible "outliers" in the data. Then they remove the outliers and ...
1
vote
0answers
35 views

What are the methods for multivariate outlier detection? [duplicate]

I have to work with research paper using multivariate outlier detection methods. Please list the methods for multivariate outlier detection and give some suggestions and ideas....
0
votes
2answers
93 views

What if a log transformation wipes out significance in regression?

I'm doing an OLS regression of donations made by individuals to a not-for-profit organisation. The donation amount is the dependent variable and dummy (treatment) variables are the only independent ...
3
votes
3answers
96 views

Outlier detection in very small sets

I need to get as accurate as possible a value for the brightness of a mainly stable light source given twelve sample luminosity values. The sensor is imperfect, and the light can occasionally ...
0
votes
1answer
182 views

Dixon Q Test - can I (all by myself) calculate the value of the Q critical?

Im just wondering .. Everywhere I see only tables with the Qcritic as the results. Lets say I have a big set of data samples (like 1000 or more). Is then a way of mathematically calculate a Qcritic ...
17
votes
7answers
788 views

Replacing outliers with mean

This question was asked by my friend who is not internet savvy. I've no statistics background and I've been searching around internet for this question. The question is : is it possible to replace ...
3
votes
2answers
153 views

Detecting outliers using correlogram

If there is an outlier in a time series, how does its correlogram behave? Is it possible to find outliers using a correlogram? EDIT I have such a Time series: ...
0
votes
0answers
30 views

Detecting outliers using periodogram?

I would like to find outliers in a time series. If there is a an outlier in a time series, how does its periodogram behave? For example I have a time series with 10 elements, and I think 515 in this ...
0
votes
1answer
83 views

Quick and easy way to remove outliers

I have a set of data of locations and associated rent prices. Now there seem to be several outliers which I would like to get rid of so that a plot of my original data gains more meaning. In the ...
0
votes
1answer
69 views

Robustly standardize residuals in MM regression

Does anyone know how we can robustly standardize the residuals in MM regression? First we perform MM regression and then obtain the residuals: how can we robustly standardize the residuals obtained ...
2
votes
1answer
68 views

Detecting outlier cash movements

If I'm watching a series of accounts for transactions going in and transactions going out, I want to notice unusually large or transactions for any particular account on any particular day. So if ...
1
vote
0answers
322 views

Outlier detection and smoothing for multi dimensional time series

From kinect depth images, I have collected the following time series that represent the features = 3D joint positions, quarternion angles, difference between hip joint and centroid of the right arm ...
1
vote
1answer
126 views

Finding anomalies using moving average in a time series [duplicate]

I want to find anomalies in a time series. Is it possible to find anomalies using moving average?
1
vote
1answer
78 views

Moving average filter for outlier removal

I am using a moving average filter to smooth data for outlier removal. By changing the number of average points, I am getting different result. My data are multi-dimensional feature vectors. I ...
6
votes
0answers
154 views

Generalized Linear Mixed Models: Diagnostics

I have a random intercept logistic regression (due to repeated measurements) and I would like to do some diagnostics, specifically concerning outliers and influential observations. I looked at ...