Skip to main content

Questions tagged [mad]

Median Absolute Deviation (MAD) is a measure of variability in a sample of data, and is often used as an alternative to measures like standard deviation since it is more resistant to outliers. Use the [mae] tag if you are asking about the point forecast accuracy measure called MAD or MAE.

Filter by
Sorted by
Tagged with
3 votes
2 answers
305 views

Is the exponent of the MAD (Median Absolute Deviation) of log transformed Data measuring the relative distance from median in the untransformed data?

I want to confirm whether taking the Exponent of the MAD of Log Transformed Data gives me a measure of relative distance from median of the original untransformed data. So say I have a MAD of 0.2 for ...
Anon9001's user avatar
1 vote
0 answers
65 views

Converting median absolute deviation to standard deviation

I'm working on some medical research and I'm wondering what is the most straightforward method to convert median and median absolute deviation (MAD) to mean and standard deviation. If anyone has any ...
Yomna's user avatar
  • 21
1 vote
1 answer
47 views

The Rousseeuw-Croux scale estimators and the Mean Absolute Deviation (MeanAD)

The 1993 Rousseeuw-Croux scale estimators were introduced as alternatives to the Median Absolute Deviation (MedianAD), by saying that: (1) Of course, there is nothing to stop us from using the ...
Ommo's user avatar
  • 270
0 votes
0 answers
274 views

Approximating Median Absolute Deviation (MAD) with Rolling Median for Normalization: Trade-offs?

I'm working with a time series dataset and am interested in normalizing the data using the rolling Median Absolute Deviation (MAD). The true MAD is defined as: $$ \text{MAD} = \text{rolling median}(|...
The One's user avatar
  • 205
2 votes
1 answer
231 views

How to calculate median and median absolute deviation of data with a function applied?

I have a set of data X containing members xi (these are pixel values). I need to calculate the median and MAD of f(X), where f is a color space transform from linear RGB to sRGB (i.e. f(x) = x^(1/2.2))...
Adrian K-B.'s user avatar
1 vote
1 answer
110 views

pros and cons of different robust measures of scale/ dispersion

I would very much appreciate some help regarding how to interpret different robust measures of scale (Inter-quartile range or IQR, biweight midvariance, and median absolute deviation or MAD). Thus, ...
user222456's user avatar
1 vote
0 answers
83 views

Is Centering Data Around Their Medians in Least Absolute Deviation Regression Model (No Intercept), a Good Robust Practice For Smaller Data Sets?

Per the regression model: $\mathbf{y} = f(\mathbf{x},\mathbf{\beta}) + \mathbf{\epsilon}$ Where the $\beta$ estimate of LAD regression is given by: $ \hat{\beta}_{LAD} = \text{argmin}_{ b} \sum_{i=1}^...
AJKOER's user avatar
  • 2,288
0 votes
1 answer
30 views

Two different ways of calculating MAD return different solutions

I have a large dataframe data with following columns: ...
jstaxlin's user avatar
  • 221
0 votes
0 answers
237 views

Median, MAD Normalization

I am trying to normalize prices based on the (median, MAD) transformation. I attach a image below describing it in comparison to the (mean, std) normalization. I have trouble underatanding if ...
Simon Rydstedt's user avatar
1 vote
1 answer
262 views

Mean distance from center of multivariate Gaussian distribution

Say I have a multivariate Gaussian distribution and I want to measure the expected vector distance of points from the mean of this distribution. If I know the covariance matrix of my mvtgaussian, is ...
user avatar
1 vote
0 answers
113 views

MAD & Median of weighted GMM

What is the median and median-absolute-deviation of a weighted GMM in terms of component mean and variance? For example, three normal distributions $A$, $B$, $C$ with means $\mu_a,\mu_b,\mu_c$, ...
user19087's user avatar
  • 123
0 votes
0 answers
95 views

Constructing a User Profile for Music Taste

My goal is to construct User profiles based on positive (and maybe also negative) interactions with songs. A User has the option to like a song. This would give me a list of likes for each user. With ...
Flitschi's user avatar
1 vote
1 answer
281 views

Method for outlier detection in noisy seasonal time series data?

I have around 1000 times series of around 1000 samples, where each sample is 5 minutes a part. An example of a time series after performing seasonal decomposition is As we can see the data is very ...
kspr's user avatar
  • 171
0 votes
1 answer
76 views

Mean and Mean Absolute Deviation inferential statistics

Consider the following data of runners in a race. The first data is from a certain year the second one is from the year after. $Last\space year \space|\space Mean: 525s \space|\space MAD: 25s$ $This \...
Mush-A's user avatar
  • 105
2 votes
1 answer
11k views

Calculating robust z scores with median and MAD

Could someone explain the scaling factors involved in calculating robust z scores using median and MAD please? As I understand it, conventional Z scores calculated using the mean and SD are sensitive ...
Owen's user avatar
  • 23
3 votes
1 answer
3k views

Median absolute deviation only can be used for anomaly detection for time series without a trend?

I think MAD only can be used to detect anomalies for time series without a trend because it relies only a stable median to detect anomalies. It should be OK for time series with seasonality. Just seek ...
etang's user avatar
  • 977
2 votes
2 answers
2k views

Replacing outlier removal from IQR to MAD

A common outlier removal formula is Q3 + IQR * 1.5 and Q1 - IQR * 1.5 Outliers can also be removed using Mean Absolute Deviation and Median Absolute Deviation. Is ...
Brad's user avatar
  • 600
1 vote
0 answers
41 views

Check which parameter causes less dispersion in a sample set - When to use CV (Coefficient of Variation) or CD (Coefficient of Dispersion) etc

The overall goal is to extract/engineer features from approximately 100 segments (various lengths but always more than 80 data points) that are as similar as possible to each other and have a very low ...
Checker9's user avatar
2 votes
0 answers
227 views

rolling removing outliers: include or not include

In the paper "Realized kernels in practice: trades and quotes" by O. E.Bandorff-Nielsen etc. cf. https://onlinelibrary.wiley.com/doi/full/10.1111/j.1368-423X.2008.00275.x in the section dedicated to ...
ABK's user avatar
  • 556
1 vote
0 answers
62 views

Mean Absolute Deviation and data preprocessing

Assume we have data points $x_{1}, \dots, x_{n}, x_{n+1}$. Next, based on Mean Absolute Deviation (MAD) we aim to decide if the last point $x_{n+1}$ is outlier or not. First, let us compute the MAD: ...
ABK's user avatar
  • 556
0 votes
0 answers
721 views

Convert median absolute deviation (MAD) to SD for log-normal distribution

How do I convert the $\text{MAD}$ (median absolute deviation from the median) of data that is drawn from a log-normal distribution to the standard deviation of a log-normal distribution? To clarify, ...
rp1's user avatar
  • 41
1 vote
3 answers
2k views

What's the best method to measure relative variability for non normal data?

I have a set of data that states the volume required each month for the next 12 for 750 raw materials. I would like to determine the variability in the demand for each material, and categorise the ...
Shaun's user avatar
  • 11
1 vote
1 answer
99 views

Inference using robust statistics

I'm simulating the overall usage of a cluster using historic deployment data. Due to the nature of the simulation, there are some heavy points (i.e. very low overall usage). As a result, the variance ...
David Lehnherr's user avatar
0 votes
1 answer
461 views

Welch t-test using the median and MAD

Can the Welch t-test be used with the median and MAD instead of mean and variance? I think outliers are causing problems and the median places less weight on extreme outliers. I'm using this to test ...
user842807's user avatar
1 vote
0 answers
1k views

Median absolute deviation outlier detection for new data points

I have been reading this article for outlier detection and the method proposed works really well when applied over my data set. However, it would be great if i could run outlier check for new data ...
John's user avatar
  • 343
7 votes
1 answer
7k views

How to estimate the scale factor for MAD for a non-normal distribution?

I understand that the scale factor for normally distributed data is 1.4826 to convert it to a pseudo standard deviation like quantity which could be used with the median for determining confidence ...
Mr. Confused's user avatar
2 votes
1 answer
1k views

MAD & Standard Deviation

Am I correct in saying that you can only use the formula for approximating the standard deviation from MAD, i.e $$\text{SD } = K \times \text{ MAD }$$ if you know the actual probability ...
David Thompson's user avatar
3 votes
1 answer
590 views

How to prove 'For a symmetric distribution with zero mean, the population MAD is the 75th percentile of the distribution'

For a symmetric distribution with zero mean, the population MAD is the 75th percentile of the distribution I came across this statement in the wikipedia page for MAD(Median Absolute Deviation) but ...
user avatar
2 votes
0 answers
202 views

Robust Regression

The Median Absolute Deviation (MAD) is often used as a measure of dispersion of the residuals in robust regression. For univariate data, MAD is computed by first finding the median of the data, then ...
Ike's user avatar
  • 21
4 votes
0 answers
574 views

Interpretation of the least absolute deviations linear regression coefficient

In linear regression of $y$ onto $x$, one finds a $\beta_0$ and $\beta_1$ minimizing $\sum \|y - (\beta_1 x + \beta_0)\|^2$. One can show that $$\beta_1 = \rho(x,y) \frac{\sigma(y)}{\sigma(x)},$$ ...
Apprentice's user avatar
0 votes
1 answer
186 views

Only four values per site - how do I detect outliers (with MAD)?

I have four weights. I want to test if there is an outlier within those four weights. To me, the number seems quite small to detect outliers. I thought that the Median Absolute Deviation (MAD) ...
Vera Marya's user avatar
4 votes
1 answer
800 views

How to calculate the median absolute deviation for a continuous function?

I know that the median value $(\text{Med})$ for a variable $X$ characterised by the contentious PDF function $f_X(x)$ can be calculated by finding $\text{Med}$ in: $\int^\text{Med}_{-\infty} dF_X(x) =...
user2350366's user avatar
1 vote
1 answer
281 views

How does one calculate Fisher-consistency factor for Rousseeuw and Croux's $S_n$ for empirical distribution?

In "Alternatives to the Median Absolute Deviation" (Rousseeuw and Croux, J. Amer. Statistical Assoc, 88(424), 1993, pp.1273–1283), the authors described an estimator of SD better than median absolute ...
hans-t's user avatar
  • 569
5 votes
3 answers
4k views

How to normalize if MAD equals zero?

A known way to normalize our feature vectors is: $$\frac{x_i - \operatorname{median}( X^{(j)})}{\operatorname{MAD}^{(j)}},$$ where $\operatorname{MAD}^{(j)}$ is the median absolute deviation of ...
Low Yield Bond's user avatar
7 votes
1 answer
4k views

Can Median Absolute Deviation (MAD)/SD be used to determine if a distribution is normal or not?

I have recently come across this post on Median Absolute Deviation (MAD). The Wikipedia article here, by the article as an estimator Standard deviation of the distribution is 'k' times MAD, where the ...
Ironluca's user avatar
  • 208
0 votes
0 answers
69 views

Square of MAD as variance equivalent

I have to estimate the error of a 1D array. The usual way of doing this to take the SD of it and square it. The issue is the following: The values in the array represent the time series of the ...
Sebastiano1991's user avatar
2 votes
1 answer
467 views

Why don't dispersions like median deviation and mode deviation exists on the lines of mean deviation?

Or to say in the formula MD=$\frac{1}{N}\sum_{i=1}^{N}|x_i-\overline{x}|$ why can't we have median or mode instead of mean($\overline{x}$) and as a result speak about additional possible measures of ...
ankit's user avatar
  • 421
12 votes
4 answers
5k views

Is there a version of the correlation coefficient that is less-sensitive to outliers?

The correlation coefficient is: $$ r = \frac{\sum_k \frac{(x_k - \bar{x}) (y_k - \bar{y_k})}{s_x s_y}}{n-1} $$ The sample mean and the sample standard deviation are sensitive to outliers. As well, ...
Ms. Molly Stewart-Gallus's user avatar
1 vote
1 answer
234 views

What is the best way to determine which proteins are significantly bound on a testing chip?

I've got a question about the data from a biological experiment. Three times the same 1024 different proteins are spotted on one testing chip. Target of the experiment is to see whether certain ...
Max's user avatar
  • 13
0 votes
0 answers
2k views

What should I use to establish a threshold to detect threshold crossings?

I have data graphed in green here: I want to detect the peaks but I don't know how to define the threshold values. The blue lines are mean ± standard deviation. Using that, I would miss the three ...
SSteve's user avatar
  • 101
3 votes
0 answers
601 views

alternative to IQR [duplicate]

I am currently making a short literature study of robust and efficient estimators. Some very well known are the median absolute deviation (MAD) and the interquartile range (IQR). However they both ...
Nenunathel's user avatar
4 votes
0 answers
8k views

When the Median Absolute Deviation (MAD) is zero

Suppose my data look like the following: (10, 10, 10, 10, 10, 0) Would it be possible to remove an outlier in this distribution using the median absolute deviation? Of course, you wouldn't need to ...
Jack's user avatar
  • 49
0 votes
1 answer
73 views

Reducing density of outliers

I have a dataset with right skewed data. It represent frequency of candidates vs their TTB (which is essentially the number of days) Now the TTB values less than 14 or so are possible. But not with ...
learn_code's user avatar
10 votes
1 answer
36k views

Determine outliers using IQR or standard deviation?

Similar to ... which doesn't have the answer I'm looking for. The data set has a normal distribution. For the project I'm working on, outliers have to be determined over residuals of breeding data....
Bas's user avatar
  • 223
5 votes
1 answer
2k views

Median absolute deviation: impact/bias when forcing a median of zero

I want to estimate the median absolute deviation (MAD) of a signal. The MAD is defined as the median of the absolute difference between the signal and its median. Now I have a signal that I know for ...
P-Gn's user avatar
  • 147
17 votes
2 answers
7k views

Median absolute deviation (MAD) and SD of different distributions

For normally distributed data, the standard deviation $\sigma$ and the median absolute deviation $\text{MAD}$ are related by: $\sigma=\Phi^{-1}(3/4)\cdot \text{MAD}\approx1.4826\cdot\text{MAD},$ ...
vic's user avatar
  • 171
15 votes
1 answer
412 views

Are there non-trivial settings where the MAD statistic has a closed-form density?

The MAD statistic of an iid sample $(x_1,\ldots,x_n)$ is defined as the median of the absolute deviation from the median: $$ \text{mad}(x_1,\ldots,x_n)=\text{med}\left\{|x_i-\text{med}(x_1,\ldots,x_n)|...
Xi'an's user avatar
  • 106k
19 votes
2 answers
14k views

MAD formula for outlier detection

Does anyone know what is the name of this formula? $$M_i = \displaystyle\frac{0.6745(x_i - \hat{x})}{\mathrm{MAD}}$$ where $\textrm{MAD}$ is the median absolute deviation and $\hat{x}$ is the median ...
synonym's user avatar
  • 193
1 vote
1 answer
115 views

Better than Mad, Derive $Q_n$

Could someone be so kind as to walk me through how to do this equation on a dataset? on pg 5 (marked as 1277) equation 3.3 of this paper. $$Q_n = c|x_i-x_j|_{(k)},\;\;1\leq i<j\leq n,k\approx{n\...
thistleknot's user avatar
0 votes
0 answers
2k views

Bar plot with median +/- median absolute deviation in scientific publication

I know bar plots usually represents mean +/- sem, but if the data is non-normal, is it ok to bar plot median +/- mad? I don't like box plots. Suggestison?
user3236594's user avatar