Skip to main content

Questions tagged [outliers]

An outlier is an observation that appears to be unusual or not well described relative to a simple characterization of a dataset. A discomfiting possibility is that these data come from a different population than the one intended to be studied.

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

Box plot: Easiest way to detect anomalies? [closed]

Is box plot an easiest way to detect anomalies making use of the popular way Q1 - 1.5 x IQR for lower bound and Q3 + 1.5 x IQR for upper bound? How often this 1.5 multiple changed by the statisticians....
Splendid Digital Solutions's user avatar
2 votes
1 answer
42 views

How can I show statistically that one of my replicates is likely contaminated?

I have a dataset that looks like the below: five replicate samples, each of which is composed of 4 different fractions that sum to 100%. The fifth sample clearly looks visually distinct from the other ...
Dubukay's user avatar
  • 278
0 votes
0 answers
29 views

Determining the p-value of a test statistic, which is not distributed according to a commonly known distribution under the null hypothesis

Currently I am working in R on a project that aims to identify Dragon King events (massive outliers) in large datasets. These outliers appear for example in the city sizes in England, where London is ...
user25936873's user avatar
1 vote
1 answer
28 views

What is the interpretation of outlier-robust principal component analysis?

There's a set of methods called "robust" principal component analysis (here, "robust" means resistant to influence from outliers). One example is Hubert et al., "ROBPCA: A new ...
cgmil's user avatar
  • 1,373
1 vote
1 answer
63 views

to determine the appropriate threshold of the z-score for the non-normally distributed data

I am interested in CPI. And I need to identify outliers in the series. For that, my instructor mentioned about the number of standard deviations from the mean that a data point is. This is Z-score. I ...
1190's user avatar
  • 1,140
2 votes
2 answers
471 views

Methods for Detecting outliers in a time series

I have a question on detecting the outliers in a time series like PPI, CPI, inflation,...etc.) Which method should I use? How can I precisely detect these outliers in a test or a method? Please ...
2 votes
1 answer
38 views

Calculate the confidence that the data point is NOT explained by the regression

I have $n$ independent variables $x_i$ and dependent variables $y_i$ with uncertainties for both $x$ and $y$. I did a linear regression to get a model $\hat y = \beta x$. Now I want to use this ...
Tibor's user avatar
  • 135
1 vote
0 answers
43 views

How to deal with outliers in panel data? [closed]

When we have cross-sectional data, we can easily detect and remove outliers. But how should one approach outliers when we are dealing with panel data? Since we have $i$ entities and $t$ times periods, ...
TFT's user avatar
  • 345
2 votes
1 answer
42 views

Interpreting Mass-Volume as an evaluation criterion for unsupervised anomaly detection

I have found this paper How to Evaluate the Quality of Unsupervised Anomaly Detection Algorithms? by Nicolas Goix that talks about evaluation of unsupervised anomaly scoring functions by the use of ...
deblue's user avatar
  • 243
1 vote
1 answer
47 views

Intensity outliers/anomalies in 2D plot

I wonder what kind of method better to use to see outliers on z value of 2D plot. For example, I have measurements of x and y values both in range of 1 to 16 with step of 1. Next I calculate how many ...
Zoomman's user avatar
  • 11
8 votes
4 answers
913 views

Why divide data into 4 parts for IQR, and not into parts of 20 or 10 percentages each?

Why divide data into 4 parts for IQR, versus into more parts, such as 20 or 10 percent per part? I know that interquartile range by definition means 25%, but that is not my question. I think that ...
True_Nerd's user avatar
2 votes
1 answer
53 views

Maximum likelihood estimation with (robust) Huber-White standard errors appropriate for outlier management?

Is maximum likelihood estimation with robust Huber-White standard errors and a scaled test statistic — which is asymptotically equal to the Yuan-Bentler test statistic — appropriate for data with ...
Madamadam's user avatar
  • 247
0 votes
0 answers
15 views

How to compute a weighted mean as a measure of central tendency?

Imagine I have data points in triplicates, as follows $$ L=\begin{pmatrix} p_{11} & p_{12} & p_{13}\\ p_{21} & p_{22} & p_{23}\\ & \vdots &\\ p_{n1} & p_{n2} & p_{n3} \...
sam wolfe's user avatar
  • 121
0 votes
0 answers
24 views

Addressing Heteroscedasticity in Mixed Effects Models with glmmTMB and DHARMa in R [duplicate]

I am analyzing ecological data in R, where I aim to understand the impact of urbanization on species trends. My response variable is the coefficient of species trends (estimate), and my main predictor ...
Pau Colom Montojo's user avatar
0 votes
0 answers
67 views

How to deal with under-dispersion in negative binomial GLMM?

I have some animal species. I am interested in seeing what is the relationship between the area they occupy (my response variable, p, which is a count of cells) and ...
LT17's user avatar
  • 161
4 votes
1 answer
37 views

Why Grubbs test is not finding outliers?

I am new to Statistics. I am trying to find outliers in the given data using Grubbs method in python. But Grubbs is not able to find outliers for my data. ...
Selva's user avatar
  • 183
4 votes
1 answer
57 views

Removing high influence points in GAM fits

I am currently building a GAM model to describe house prices. The dataset is a collection of roughly 200K house sale prices $\{P_{i}\}$ together with a vector of house characteristics $\{\textbf{x}_{i}...
August Edwards's user avatar
0 votes
1 answer
48 views

Outliers in OLS

I'm running OLS regression in stata with 5-6 IVs (all dummy variables) and a DV (continuous). Does it make sense to check for outliers (e.g., I can't run scatter dv x1 x2) in this case? Also, is it ...
brian's user avatar
  • 75
0 votes
1 answer
40 views

question about outliers in a 5 term data set

the question states that no data set with only 5 terms has an outlier, and I'm stumped you're given a set of 5 numbers in ascending order $$ x_1, x_2, x_3, x_4, x_5 $$ I started by finding the IQR, ...
Bongo 186's user avatar
1 vote
0 answers
56 views

Valid approach: Winsorizing data for main analysis and then doing sensitivity analysis without winsorizing?

I've got a variable with psychological data (N=75) which is distributed pretty symmetrical, but has very few cases with very extreme values, more extreme to the left tail. But nevertheless this data ...
Malea Dondé's user avatar
5 votes
4 answers
629 views

Dropping outlier from linear regression model reducing adjusted R^2

I'm running a linear regression in R on a dataset with 8 independent variables. When I run the model with all variables: ...
S. Dolan's user avatar
0 votes
0 answers
33 views

How to interpret the cut-off of a Mahalanobis distance in R

Making the cut-off, I see two multivariante outlieres in the plot and get 4 numbers printet in my console. (43 and 2 in the first row and 2 and 23 in the second row) What do they stand for? How can I ...
user avatar
0 votes
0 answers
24 views

Approach for multivariate outlier detection when treating missing values with FIML

I‘m calculating a simple regression with one predictor and one dependent variable. Missings treatment is done with full information maximum likelihood (FIML). Should I do outlier detection, i.e. ...
Madamadam's user avatar
  • 247
2 votes
0 answers
44 views

One-way repeated measures ANOVA with skewed response

We have an experiment with 102 individuals in total. We have an outcome $Y$ (which is a variable related to the structure of a given bone), and we want to know whether this variable $Y$ differs ...
Leandro T.'s user avatar
0 votes
2 answers
43 views

Should you remove outliers in a small insurance dataset? [closed]

Background I'm working on an insurance dataset that has 4,000 rows, and 30 columns. The target variable is the loss incurred by the customer for the given row. The target variable's distribution looks ...
Connor's user avatar
  • 655
0 votes
0 answers
22 views

What should I do when my data is normal, but not homogen?

My data is (n:43)genotypes with block as replication (n:2). the design is randomized complete block design. and I did normality test and the result said normal, but I did homogeneity test (levenetest) ...
Nimas Pertiwi's user avatar
1 vote
1 answer
40 views

Wrong time series detection

I have a problem and I need help. I have a time series and I need to know if the data is correct. Let me explain with an example. Suppose I have data generated by an atmospheric pressure sensor. The ...
Rirro Romeu's user avatar
1 vote
0 answers
25 views

Outliers in Delta Time column, using data from wireshark

I am currently analyzing data downloaded from Wireshark, focusing on real-time network traffic. I need to perform clusterization on this dataset. However, during the visualization process, I noticed ...
Jaminka's user avatar
  • 11
6 votes
2 answers
129 views

Compare failure rates across multiple systems

I work in pharmaceutical manufacturing and one part of a process is a filtration step that uses 'clusters' or 'sets' of single-use (disposable) filters in parallel - the product flows into a manifold ...
ChemEnger's user avatar
2 votes
1 answer
54 views

How do I handle outliers?

I'm calculating the beta coefficients for some stocks using a single-index linear model with the OLS method. I'm computing the betas at different return intervals to assess the interval effect on the ...
Mattia's user avatar
  • 151
1 vote
1 answer
54 views

Removing outliers in several groups and for several features

I'm unsure on how to remove or winsorize outliers. Let's say I have 2 groups, treated and control. And I measure feature1 and feature2 for both. How should I handle outliers? For each group and each ...
Caterina's user avatar
  • 229
0 votes
0 answers
47 views

Outliers in EDA - With or without?

I'm trying to carry out my first EDA on a Student Performance dataset. The dataset has 395 samples and consists of 33 attributes. After drawing the boxplots and doing some tests I detected outliers in ...
Christina Kataki's user avatar
0 votes
1 answer
160 views

Impact of outliers to QQ plot

I'm trying to build an GLM regression (10k samples and 50 dimensions). I ran an analysis of the dependent variable since the regression has a normality assumption for the dependent variable. The QQ ...
cat's user avatar
  • 53
1 vote
0 answers
31 views

Outlier detection on a measurement stream. Decision theoretic, Bayesian approaches?

I have a stream of real valued measurements $x_1, x_2, \dotsc$ that I expect to be, for the most part, normal distributed with some unknown mean $\mu > 0$ and unknown variance $\sigma^2$. However, ...
ummg's user avatar
  • 145
1 vote
0 answers
20 views

Applying Tangent Lines to Log-Scaled Data for Outlier Detection: Seeking Statistical Theories and Models

I've analyzed the view counts for a YouTube channel's videos (just for example), sorting them by views (on the left) and drawing a tangent line to approximate the central trend on a logarithmic scale (...
Andrew Anderson's user avatar
4 votes
3 answers
284 views

Outlier detection methods aware of target variable

I am trying to predict ambulance demand for the next hour, for a city area in the USA, based on previous demand, weather, large people gatherings, and similar spatio-temporal factors. I have noticed ...
Nadir Bašić's user avatar
1 vote
1 answer
51 views

Extreme values affecting mean in regression analysis

I am examining whether there is a difference in reaction time (RT) for 2 different conditions (A and B). Participants complete trials in both conditions. First, I use a linear mixed model to assess ...
SilvaC's user avatar
  • 512
0 votes
1 answer
96 views

Outlier Detection and Removal

I am reading a paper on wind power forecasting and the authors present a plot of the data before outliers are removed and a plot after. However, they don't actually say what method was employed to ...
x0929's user avatar
  • 1
2 votes
1 answer
33 views

ARIMA - Identifying an outlier in residuals

I am trying to perform an ARIMA (SARIMAX in fact) and when looking at the residuals I see a large outlier. I am using python statsmodels.tsa.statespace.sarimax. I ...
Solebay Sharp's user avatar
1 vote
1 answer
169 views

Tukey's IQR-method for outliers and highly skewed data

I am writing a thesis on performances on cognitive and linguistic measures. I have used the Tukey IQR method (Q1-1.5*IQR) to detect lower outliers in a non-normally distributed small sample of various ...
Samplename1's user avatar
0 votes
1 answer
53 views

Identify outliers in testing data based on trained Gaussian mixture model

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
baronett's user avatar
4 votes
3 answers
280 views

How to compare influence of outlier in regression model. ANOVA of two models in R

I am doing linear regression in R. I have identified an outlier in my data: ...
Mark Davies's user avatar
1 vote
1 answer
33 views

When detecting outliers for an ANVOA test for multiple groups, should I do this for the whole population sample or by group?

I have data that is grouped by 5 groups. I want to check for outliers as part of a one-way anova. Should the check for outliers on the whole sample, or by group? I am using box plots to look for ...
Mark Davies's user avatar
1 vote
0 answers
92 views

Evaluate CDF and outliers of multidimensional Gaussian mixture [closed]

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
baronett's user avatar
1 vote
0 answers
29 views

Can there be no outliers as per calculation with IQR formula while the boxplot shows there are outliers in the dataset?

While plotting a box plot, the plot is showing the columns in the dataset has outliers, but while trying to calculate it by IQR formula, it is showing there are 0 outliers in the columns of the ...
Taniya Pal's user avatar
0 votes
0 answers
33 views

Normalization/standardization of time series data

I have energy consumption data where rows represent different users and columns are different measurements. I don't really understand, how and in which order i need to normalize and standardize the ...
Moulmein's user avatar
0 votes
1 answer
71 views

Hypothesis testing - Newbie blockers - Update and more

Brief : I'm from manufacturing industry, a processing machine in our production line used to do pressing, polishing and QA one after the other. Now we have a new machine that will perform these at the ...
AKK's user avatar
  • 3
1 vote
0 answers
31 views

BEST POSSIBLE WAY to determine significantly high values within zero-inflated univariate continuous distributions

I have more than 50 different distributions, corresponding to 50 different kind of customers, who spend their money in a certain way within a period, being this amount the single variable of interest. ...
0xGolovkin's user avatar
1 vote
0 answers
33 views

Outlier in three dimension [closed]

I am looking for an example of an outlier in three dimensions but that cannot be detected with the three scatters xy, xz, yz. Can anybody help me? Thanks in advance.
Augusto's user avatar
  • 21
3 votes
2 answers
55 views

Can using the IQR/Median help with this problem?

I have a set of data in tabular form which records the time it takes 500 people to bake a cake. Each person is assigned a single time: it may take Tom 30 minutes, or Mike 60 minutes to bake a cake. ...
Chips220 Swifty's user avatar

1
2 3 4 5
28