Skip to main content

Questions tagged [outliers]

An outlier is an observation that appears to be unusual or not well described relative to a simple characterization of a dataset. A discomfiting possibility is that these data come from a different population than the one intended to be studied.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

What is the interpretation of outlier-robust principal component analysis?

There's a set of methods called "robust" principal component analysis (here, "robust" means resistant to influence from outliers). One example is Hubert et al., "ROBPCA: A new ...
cgmil's user avatar
  • 1,353
1 vote
1 answer
46 views

to determine the appropriate threshold of the z-score for the non-normally distributed data

I am interested in CPI. And I need to identify outliers in the series. For that, my instructor mentioned about the number of standard deviations from the mean that a data point is. This is Z-score. I ...
1190's user avatar
  • 1,140
2 votes
2 answers
438 views

Methods for Detecting outliers in a time series

I have a question on detecting the outliers in a time series like PPI, CPI, inflation,...etc.) Which method should I use? How can I precisely detect these outliers in a test or a method? Please ...
1190's user avatar
  • 1,140
2 votes
1 answer
37 views

Calculate the confidence that the data point is NOT explained by the regression

I have $n$ independent variables $x_i$ and dependent variables $y_i$ with uncertainties for both $x$ and $y$. I did a linear regression to get a model $\hat y = \beta x$. Now I want to use this ...
Tibor's user avatar
  • 135
1 vote
0 answers
37 views

How to deal with outliers in panel data? [closed]

When we have cross-sectional data, we can easily detect and remove outliers. But how should one approach outliers when we are dealing with panel data? Since we have $i$ entities and $t$ times periods, ...
TFT's user avatar
  • 345
2 votes
1 answer
39 views

Interpreting Mass-Volume as an evaluation criterion for unsupervised anomaly detection

I have found this paper How to Evaluate the Quality of Unsupervised Anomaly Detection Algorithms? by Nicolas Goix that talks about evaluation of unsupervised anomaly scoring functions by the use of ...
deblue's user avatar
  • 243
1 vote
1 answer
45 views

Intensity outliers/anomalies in 2D plot

I wonder what kind of method better to use to see outliers on z value of 2D plot. For example, I have measurements of x and y values both in range of 1 to 16 with step of 1. Next I calculate how many ...
Zoomman's user avatar
  • 11
7 votes
4 answers
903 views

Why divide data into 4 parts for IQR, and not into parts of 20 or 10 percentages each?

Why divide data into 4 parts for IQR, versus into more parts, such as 20 or 10 percent per part? I know that interquartile range by definition means 25%, but that is not my question. I think that ...
True_Nerd's user avatar
2 votes
1 answer
44 views

Maximum likelihood estimation with (robust) Huber-White standard errors appropriate for outlier management?

Is maximum likelihood estimation with robust Huber-White standard errors and a scaled test statistic — which is asymptotically equal to the Yuan-Bentler test statistic — appropriate for data with ...
Madamadam's user avatar
  • 247
0 votes
0 answers
15 views

How to compute a weighted mean as a measure of central tendency?

Imagine I have data points in triplicates, as follows $$ L=\begin{pmatrix} p_{11} & p_{12} & p_{13}\\ p_{21} & p_{22} & p_{23}\\ & \vdots &\\ p_{n1} & p_{n2} & p_{n3} \...
sam wolfe's user avatar
  • 121
0 votes
0 answers
23 views

Addressing Heteroscedasticity in Mixed Effects Models with glmmTMB and DHARMa in R [duplicate]

I am analyzing ecological data in R, where I aim to understand the impact of urbanization on species trends. My response variable is the coefficient of species trends (estimate), and my main predictor ...
Pau Colom Montojo's user avatar
0 votes
0 answers
53 views

How to deal with under-dispersion in negative binomial GLMM?

I have some animal species. I am interested in seeing what is the relationship between the area they occupy (my response variable, p, which is a count of cells) and ...
LT17's user avatar
  • 161
4 votes
1 answer
32 views

Why Grubbs test is not finding outliers?

I am new to Statistics. I am trying to find outliers in the given data using Grubbs method in python. But Grubbs is not able to find outliers for my data. ...
Selva's user avatar
  • 183
4 votes
1 answer
51 views

Removing high influence points in GAM fits

I am currently building a GAM model to describe house prices. The dataset is a collection of roughly 200K house sale prices $\{P_{i}\}$ together with a vector of house characteristics $\{\textbf{x}_{i}...
August Edwards's user avatar
0 votes
1 answer
47 views

Outliers in OLS

I'm running OLS regression in stata with 5-6 IVs (all dummy variables) and a DV (continuous). Does it make sense to check for outliers (e.g., I can't run scatter dv x1 x2) in this case? Also, is it ...
brian's user avatar
  • 75
0 votes
1 answer
39 views

question about outliers in a 5 term data set

the question states that no data set with only 5 terms has an outlier, and I'm stumped you're given a set of 5 numbers in ascending order $$ x_1, x_2, x_3, x_4, x_5 $$ I started by finding the IQR, ...
Bongo 186's user avatar
1 vote
0 answers
53 views

Valid approach: Winsorizing data for main analysis and then doing sensitivity analysis without winsorizing?

I've got a variable with psychological data (N=75) which is distributed pretty symmetrical, but has very few cases with very extreme values, more extreme to the left tail. But nevertheless this data ...
Malea Dondé's user avatar
5 votes
4 answers
587 views

Dropping outlier from linear regression model reducing adjusted R^2

I'm running a linear regression in R on a dataset with 8 independent variables. When I run the model with all variables: ...
S. Dolan's user avatar
0 votes
0 answers
29 views

How to interpret the cut-off of a Mahalanobis distance in R

Making the cut-off, I see two multivariante outlieres in the plot and get 4 numbers printet in my console. (43 and 2 in the first row and 2 and 23 in the second row) What do they stand for? How can I ...
user avatar
0 votes
0 answers
22 views

Approach for multivariate outlier detection when treating missing values with FIML

I‘m calculating a simple regression with one predictor and one dependent variable. Missings treatment is done with full information maximum likelihood (FIML). Should I do outlier detection, i.e. ...
Madamadam's user avatar
  • 247
2 votes
0 answers
44 views

One-way repeated measures ANOVA with skewed response

We have an experiment with 102 individuals in total. We have an outcome $Y$ (which is a variable related to the structure of a given bone), and we want to know whether this variable $Y$ differs ...
Leandro T.'s user avatar
0 votes
2 answers
42 views

Should you remove outliers in a small insurance dataset? [closed]

Background I'm working on an insurance dataset that has 4,000 rows, and 30 columns. The target variable is the loss incurred by the customer for the given row. The target variable's distribution looks ...
Connor's user avatar
  • 635
0 votes
0 answers
20 views

What should I do when my data is normal, but not homogen?

My data is (n:43)genotypes with block as replication (n:2). the design is randomized complete block design. and I did normality test and the result said normal, but I did homogeneity test (levenetest) ...
Nimas Pertiwi's user avatar
1 vote
1 answer
40 views

Wrong time series detection

I have a problem and I need help. I have a time series and I need to know if the data is correct. Let me explain with an example. Suppose I have data generated by an atmospheric pressure sensor. The ...
Rirro Romeu's user avatar
1 vote
0 answers
24 views

Outliers in Delta Time column, using data from wireshark

I am currently analyzing data downloaded from Wireshark, focusing on real-time network traffic. I need to perform clusterization on this dataset. However, during the visualization process, I noticed ...
Jaminka's user avatar
  • 11
6 votes
2 answers
118 views

Compare failure rates across multiple systems

I work in pharmaceutical manufacturing and one part of a process is a filtration step that uses 'clusters' or 'sets' of single-use (disposable) filters in parallel - the product flows into a manifold ...
ChemEnger's user avatar
2 votes
1 answer
54 views

How do I handle outliers?

I'm calculating the beta coefficients for some stocks using a single-index linear model with the OLS method. I'm computing the betas at different return intervals to assess the interval effect on the ...
Mattia's user avatar
  • 151
1 vote
1 answer
51 views

Removing outliers in several groups and for several features

I'm unsure on how to remove or winsorize outliers. Let's say I have 2 groups, treated and control. And I measure feature1 and feature2 for both. How should I handle outliers? For each group and each ...
Caterina's user avatar
  • 229
0 votes
0 answers
46 views

Outliers in EDA - With or without?

I'm trying to carry out my first EDA on a Student Performance dataset. The dataset has 395 samples and consists of 33 attributes. After drawing the boxplots and doing some tests I detected outliers in ...
Christina Kataki's user avatar
0 votes
1 answer
155 views

Impact of outliers to QQ plot

I'm trying to build an GLM regression (10k samples and 50 dimensions). I ran an analysis of the dependent variable since the regression has a normality assumption for the dependent variable. The QQ ...
cat's user avatar
  • 53
1 vote
0 answers
31 views

Outlier detection on a measurement stream. Decision theoretic, Bayesian approaches?

I have a stream of real valued measurements $x_1, x_2, \dotsc$ that I expect to be, for the most part, normal distributed with some unknown mean $\mu > 0$ and unknown variance $\sigma^2$. However, ...
ummg's user avatar
  • 145
1 vote
0 answers
20 views

Applying Tangent Lines to Log-Scaled Data for Outlier Detection: Seeking Statistical Theories and Models

I've analyzed the view counts for a YouTube channel's videos (just for example), sorting them by views (on the left) and drawing a tangent line to approximate the central trend on a logarithmic scale (...
Andrew Anderson's user avatar
4 votes
3 answers
270 views

Outlier detection methods aware of target variable

I am trying to predict ambulance demand for the next hour, for a city area in the USA, based on previous demand, weather, large people gatherings, and similar spatio-temporal factors. I have noticed ...
Nadir Bašić's user avatar
1 vote
1 answer
49 views

Extreme values affecting mean in regression analysis

I am examining whether there is a difference in reaction time (RT) for 2 different conditions (A and B). Participants complete trials in both conditions. First, I use a linear mixed model to assess ...
SilvaC's user avatar
  • 512
0 votes
1 answer
96 views

Outlier Detection and Removal

I am reading a paper on wind power forecasting and the authors present a plot of the data before outliers are removed and a plot after. However, they don't actually say what method was employed to ...
x0929's user avatar
  • 1
2 votes
1 answer
30 views

ARIMA - Identifying an outlier in residuals

I am trying to perform an ARIMA (SARIMAX in fact) and when looking at the residuals I see a large outlier. I am using python statsmodels.tsa.statespace.sarimax. I ...
Solebay Sharp's user avatar
1 vote
1 answer
152 views

Tukey's IQR-method for outliers and highly skewed data

I am writing a thesis on performances on cognitive and linguistic measures. I have used the Tukey IQR method (Q1-1.5*IQR) to detect lower outliers in a non-normally distributed small sample of various ...
Samplename1's user avatar
0 votes
1 answer
51 views

Identify outliers in testing data based on trained Gaussian mixture model

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
baronett's user avatar
4 votes
3 answers
266 views

How to compare influence of outlier in regression model. ANOVA of two models in R

I am doing linear regression in R. I have identified an outlier in my data: ...
Mark Davies's user avatar
1 vote
1 answer
31 views

When detecting outliers for an ANVOA test for multiple groups, should I do this for the whole population sample or by group?

I have data that is grouped by 5 groups. I want to check for outliers as part of a one-way anova. Should the check for outliers on the whole sample, or by group? I am using box plots to look for ...
Mark Davies's user avatar
1 vote
0 answers
86 views

Evaluate CDF and outliers of multidimensional Gaussian mixture [closed]

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
baronett's user avatar
1 vote
0 answers
27 views

Can there be no outliers as per calculation with IQR formula while the boxplot shows there are outliers in the dataset?

While plotting a box plot, the plot is showing the columns in the dataset has outliers, but while trying to calculate it by IQR formula, it is showing there are 0 outliers in the columns of the ...
Taniya Pal's user avatar
0 votes
0 answers
32 views

Normalization/standardization of time series data

I have energy consumption data where rows represent different users and columns are different measurements. I don't really understand, how and in which order i need to normalize and standardize the ...
Moulmein's user avatar
0 votes
1 answer
70 views

Hypothesis testing - Newbie blockers - Update and more

Brief : I'm from manufacturing industry, a processing machine in our production line used to do pressing, polishing and QA one after the other. Now we have a new machine that will perform these at the ...
AKK's user avatar
  • 3
1 vote
0 answers
29 views

BEST POSSIBLE WAY to determine significantly high values within zero-inflated univariate continuous distributions

I have more than 50 different distributions, corresponding to 50 different kind of customers, who spend their money in a certain way within a period, being this amount the single variable of interest. ...
0xGolovkin's user avatar
1 vote
0 answers
33 views

Outlier in three dimension [closed]

I am looking for an example of an outlier in three dimensions but that cannot be detected with the three scatters xy, xz, yz. Can anybody help me? Thanks in advance.
Augusto's user avatar
  • 21
2 votes
2 answers
51 views

Can using the IQR/Median help with this problem?

I have a set of data in tabular form which records the time it takes 500 people to bake a cake. Each person is assigned a single time: it may take Tom 30 minutes, or Mike 60 minutes to bake a cake. ...
Chips220 Swifty's user avatar
0 votes
0 answers
13 views

Weighted count of distinct items based on their frequency in a list

I have a list of items. Each item has some properties associated with them. For example, let's consider country. Let's say out of 100 entries, 98 are from US and 2 are from UK. When considering the ...
Registered User's user avatar
0 votes
0 answers
19 views

Statistical procedure to remove a time series which is an "outlier" in a set of replicated time series

This data are from measuring optical density in a bacterial growth experiment. These correspond to 4 time series which are biological replicates of exactly the same treatment (with the label 0_4) The ...
rgvalenciaalbornoz's user avatar
2 votes
2 answers
423 views

Understanding slope in plot of residuals vs. fitted values

How should I interpret the negative relationship between the residual and the fitted value in the plot shown below?
CCZ's user avatar
  • 21

1
2 3 4 5
28