# Questions tagged [outliers]

An outlier is an observation that appears to be unusual or not well described relative to a simple characterization of a dataset. A discomfiting possibility is that these data come from a different population than the one intended to be studied.

1,354 questions
Filter by
Sorted by
Tagged with
11 views

### What is the interpretation of outlier-robust principal component analysis?

There's a set of methods called "robust" principal component analysis (here, "robust" means resistant to influence from outliers). One example is Hubert et al., "ROBPCA: A new ...
• 1,353
1 vote
46 views

### to determine the appropriate threshold of the z-score for the non-normally distributed data

I am interested in CPI. And I need to identify outliers in the series. For that, my instructor mentioned about the number of standard deviations from the mean that a data point is. This is Z-score. I ...
• 1,140
438 views

### Methods for Detecting outliers in a time series

I have a question on detecting the outliers in a time series like PPI, CPI, inflation,...etc.) Which method should I use? How can I precisely detect these outliers in a test or a method? Please ...
• 1,140
37 views

### Calculate the confidence that the data point is NOT explained by the regression

I have $n$ independent variables $x_i$ and dependent variables $y_i$ with uncertainties for both $x$ and $y$. I did a linear regression to get a model $\hat y = \beta x$. Now I want to use this ...
• 135
1 vote
37 views

### How to deal with outliers in panel data? [closed]

When we have cross-sectional data, we can easily detect and remove outliers. But how should one approach outliers when we are dealing with panel data? Since we have $i$ entities and $t$ times periods, ...
• 345
39 views

### Interpreting Mass-Volume as an evaluation criterion for unsupervised anomaly detection

I have found this paper How to Evaluate the Quality of Unsupervised Anomaly Detection Algorithms? by Nicolas Goix that talks about evaluation of unsupervised anomaly scoring functions by the use of ...
• 243
1 vote
45 views

### Intensity outliers/anomalies in 2D plot

I wonder what kind of method better to use to see outliers on z value of 2D plot. For example, I have measurements of x and y values both in range of 1 to 16 with step of 1. Next I calculate how many ...
• 11
903 views

### Why divide data into 4 parts for IQR, and not into parts of 20 or 10 percentages each?

Why divide data into 4 parts for IQR, versus into more parts, such as 20 or 10 percent per part? I know that interquartile range by definition means 25%, but that is not my question. I think that ...
44 views

### Maximum likelihood estimation with (robust) Huber-White standard errors appropriate for outlier management?

Is maximum likelihood estimation with robust Huber-White standard errors and a scaled test statistic — which is asymptotically equal to the Yuan-Bentler test statistic — appropriate for data with ...
• 247
15 views

1 vote
53 views

### Valid approach: Winsorizing data for main analysis and then doing sensitivity analysis without winsorizing?

I've got a variable with psychological data (N=75) which is distributed pretty symmetrical, but has very few cases with very extreme values, more extreme to the left tail. But nevertheless this data ...
587 views

### Dropping outlier from linear regression model reducing adjusted R^2

I'm running a linear regression in R on a dataset with 8 independent variables. When I run the model with all variables: ...
• 53
29 views

### How to interpret the cut-off of a Mahalanobis distance in R

Making the cut-off, I see two multivariante outlieres in the plot and get 4 numbers printet in my console. (43 and 2 in the first row and 2 and 23 in the second row) What do they stand for? How can I ...
22 views

### Approach for multivariate outlier detection when treating missing values with FIML

I‘m calculating a simple regression with one predictor and one dependent variable. Missings treatment is done with full information maximum likelihood (FIML). Should I do outlier detection, i.e. ...
• 247
44 views

### One-way repeated measures ANOVA with skewed response

We have an experiment with 102 individuals in total. We have an outcome $Y$ (which is a variable related to the structure of a given bone), and we want to know whether this variable $Y$ differs ...
• 359
42 views

### Should you remove outliers in a small insurance dataset? [closed]

Background I'm working on an insurance dataset that has 4,000 rows, and 30 columns. The target variable is the loss incurred by the customer for the given row. The target variable's distribution looks ...
• 635
20 views

### What should I do when my data is normal, but not homogen?

My data is (n:43)genotypes with block as replication (n:2). the design is randomized complete block design. and I did normality test and the result said normal, but I did homogeneity test (levenetest) ...
1 vote
40 views

### Wrong time series detection

I have a problem and I need help. I have a time series and I need to know if the data is correct. Let me explain with an example. Suppose I have data generated by an atmospheric pressure sensor. The ...
• 131
1 vote
24 views

### Outliers in Delta Time column, using data from wireshark

I am currently analyzing data downloaded from Wireshark, focusing on real-time network traffic. I need to perform clusterization on this dataset. However, during the visualization process, I noticed ...
• 11
118 views

### Compare failure rates across multiple systems

I work in pharmaceutical manufacturing and one part of a process is a filtration step that uses 'clusters' or 'sets' of single-use (disposable) filters in parallel - the product flows into a manifold ...
54 views

### How do I handle outliers?

I'm calculating the beta coefficients for some stocks using a single-index linear model with the OLS method. I'm computing the betas at different return intervals to assess the interval effect on the ...
• 151
1 vote
51 views

### Removing outliers in several groups and for several features

I'm unsure on how to remove or winsorize outliers. Let's say I have 2 groups, treated and control. And I measure feature1 and feature2 for both. How should I handle outliers? For each group and each ...
• 229
46 views

### Outliers in EDA - With or without?

I'm trying to carry out my first EDA on a Student Performance dataset. The dataset has 395 samples and consists of 33 attributes. After drawing the boxplots and doing some tests I detected outliers in ...
155 views

### Impact of outliers to QQ plot

I'm trying to build an GLM regression (10k samples and 50 dimensions). I ran an analysis of the dependent variable since the regression has a normality assumption for the dependent variable. The QQ ...
• 53
1 vote
31 views

### Outlier detection on a measurement stream. Decision theoretic, Bayesian approaches?

I have a stream of real valued measurements $x_1, x_2, \dotsc$ that I expect to be, for the most part, normal distributed with some unknown mean $\mu > 0$ and unknown variance $\sigma^2$. However, ...
• 145
1 vote
20 views

### Applying Tangent Lines to Log-Scaled Data for Outlier Detection: Seeking Statistical Theories and Models

I've analyzed the view counts for a YouTube channel's videos (just for example), sorting them by views (on the left) and drawing a tangent line to approximate the central trend on a logarithmic scale (...
270 views

### Outlier detection methods aware of target variable

I am trying to predict ambulance demand for the next hour, for a city area in the USA, based on previous demand, weather, large people gatherings, and similar spatio-temporal factors. I have noticed ...
1 vote
49 views

### Extreme values affecting mean in regression analysis

I am examining whether there is a difference in reaction time (RT) for 2 different conditions (A and B). Participants complete trials in both conditions. First, I use a linear mixed model to assess ...
• 512
96 views

### Outlier Detection and Removal

I am reading a paper on wind power forecasting and the authors present a plot of the data before outliers are removed and a plot after. However, they don't actually say what method was employed to ...
30 views

### ARIMA - Identifying an outlier in residuals

I am trying to perform an ARIMA (SARIMAX in fact) and when looking at the residuals I see a large outlier. I am using python statsmodels.tsa.statespace.sarimax. I ...
1 vote
152 views

### Tukey's IQR-method for outliers and highly skewed data

I am writing a thesis on performances on cognitive and linguistic measures. I have used the Tukey IQR method (Q1-1.5*IQR) to detect lower outliers in a non-normally distributed small sample of various ...
51 views

### Identify outliers in testing data based on trained Gaussian mixture model

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
• 11
266 views

### How to compare influence of outlier in regression model. ANOVA of two models in R

I am doing linear regression in R. I have identified an outlier in my data: ...
• 205
1 vote
31 views

### When detecting outliers for an ANVOA test for multiple groups, should I do this for the whole population sample or by group?

I have data that is grouped by 5 groups. I want to check for outliers as part of a one-way anova. Should the check for outliers on the whole sample, or by group? I am using box plots to look for ...
• 205
1 vote
86 views

### Evaluate CDF and outliers of multidimensional Gaussian mixture [closed]

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
• 11
1 vote
27 views

### Can there be no outliers as per calculation with IQR formula while the boxplot shows there are outliers in the dataset?

While plotting a box plot, the plot is showing the columns in the dataset has outliers, but while trying to calculate it by IQR formula, it is showing there are 0 outliers in the columns of the ...
32 views

### Normalization/standardization of time series data

I have energy consumption data where rows represent different users and columns are different measurements. I don't really understand, how and in which order i need to normalize and standardize the ...
70 views

### Hypothesis testing - Newbie blockers - Update and more

Brief : I'm from manufacturing industry, a processing machine in our production line used to do pressing, polishing and QA one after the other. Now we have a new machine that will perform these at the ...
• 3
1 vote
29 views

### BEST POSSIBLE WAY to determine significantly high values within zero-inflated univariate continuous distributions

I have more than 50 different distributions, corresponding to 50 different kind of customers, who spend their money in a certain way within a period, being this amount the single variable of interest. ...
1 vote
33 views

### Outlier in three dimension [closed]

I am looking for an example of an outlier in three dimensions but that cannot be detected with the three scatters xy, xz, yz. Can anybody help me? Thanks in advance.
• 21
51 views

### Can using the IQR/Median help with this problem?

I have a set of data in tabular form which records the time it takes 500 people to bake a cake. Each person is assigned a single time: it may take Tom 30 minutes, or Mike 60 minutes to bake a cake. ...
13 views

### Weighted count of distinct items based on their frequency in a list

I have a list of items. Each item has some properties associated with them. For example, let's consider country. Let's say out of 100 entries, 98 are from US and 2 are from UK. When considering the ...