Questions tagged [mean-absolute-deviation]
Mean absolute deviation around the mean. Use [mad] for median absolute deviation around the median.
46
questions
0
votes
0
answers
21
views
What is the significance of Mean Absolute Deviance <= Standard Deviation?
I'm having trouble grasping what the STD is trying to tell me about my data.
I noticed that in the definitions
$\sigma = \sqrt{\frac{\sum_i{(x_i-\mu)^2}}{N}}$, $mad = \frac{\sum_i{\sqrt{(x_i-\mu)^2}}}{...
0
votes
0
answers
4
views
Is there an analogue for standard error of the mean based on mean absolute deviation?
We can estimate the standard error (SE) of the sample mean as the sample standard deviation divided by the square root of the number of samples, cf. https://en.wikipedia.org/wiki/Standard_error#...
1
vote
1
answer
81
views
Absolute deviation from the mean using logarithms
I am calculating the absolute deviation from the mean of a strictly positive set $\{x_1, x_2, \ldots, x_n\}$ like:
$$\left| x_i - \bar X\right|$$
My analysis makes it appropriate to work in logs ...
- 11
1
vote
1
answer
36
views
Is it incorrect to take the standard deviation of absolute values
Imagine I have a model that predicts a person’s height from a photo (just an example). Each individual has their ground-truth height (H), and the model outputs a predicted height (h). The model error ...
- 163
0
votes
0
answers
15
views
Deviation estimated by humans
I'd like to know which type of deviation humans will naturally estimate if asked to.
Like, if you give them some points on a line and ask them to estimate the "plus or minus error" around a ...
- 181
1
vote
1
answer
54
views
Mean absolute deviation (MAD) analogy to 68-95-99 rule
With MAD, 50% of all values fall within one absolute deviation. How many within two, and three?
- 11
5
votes
1
answer
718
views
Show that, for any real numbers a and b such that m ≤ a ≤ b or m ≥ a ≥ b, E|Y − a| ≤ E|Y − b| ,where Y be a random variable with finite expectation
Let $Y$ be a random variable with finite expectation, and $m$ be a median
of $Y;$ i.e., $P(Y \le m) \ge 1/2$ and $P(Y \ge m) \ge 1/2.$ Show that, for
any real numbers $a$ and $b$ such that $m\le a \le ...
0
votes
0
answers
18
views
Count the fraction of trials for which the absolute deviation from the mean is larger than the standard deviation?
So, I have a data set containing 99 elements each with their own associated value. I have computed the standard deviation and the mean, but I am a little stumped with how exactly I should proceed with ...
- 1
0
votes
0
answers
51
views
Deducing $L^1$ Boundary from Mahalanobis Boundary
Assume maximum likelihood estimators $a,b$ of size $p$, with corresponding estimated covariance matrices $V^a,V^b$. In fact $a,b$ are two regression coefficient vectors.
Denote $q=a-b$ the vector of ...
- 2,718
0
votes
0
answers
155
views
Standard Deviations of Detrended Data vs Original Data
I'am interested to know the opinion on this matter.
I have Data of a Financial Security. The goal is to view the Standard Deviations levels of the data to possibly see patterns of behavior of the ...
- 101
0
votes
1
answer
70
views
measure for deviation/error that is guarantueed to be smaller than the mean
I am presenting some non-negative values by using a mean value with error bars, basically something like this:
So the mean values are shown as dots and the error bars are displayed vertically, ...
- 103
1
vote
0
answers
41
views
Mean Absolute Deviation for AR(1) process
Is there closed-form solution for mean absolute deviation (MAD) for AR(1) process?
$X_t = c + \beta X_{t-1} + \epsilon_t $
$\epsilon_t \sim N(0,\sigma^{\epsilon})$
(Similar to the variance and the ...
- 11
0
votes
0
answers
88
views
Least Absolute Values Regression
I have the data:
x <- c(0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)
y <- c(2.06, 2.12, 2.32, 2.02, 2.76, 3.04, 2.83, 3.15, 3.36, 3.68, 3.96)
I ...
1
vote
0
answers
218
views
How do I calculate Correlation Coefficient using Mean Absolute Deviation?
I am trying to calculate Pearson's Correlation Coefficient using the product of the Mean Absolute Deviations (MADs) of my two lists as denominators instead of the Standard Deviations. Effectively, ...
- 11
1
vote
0
answers
57
views
Estimate the error bound of a histogram based mean absolute deviation approximation
Given a sequence of numbers $A = a_1, a_2, \cdots, a_n$.
One way to calculate the mean absolute deviation of $A$ is by $G(A)= \sum_{i=0}^n |a_i - median(A)|$.
yet, there can be an alternative, by ...
- 151
3
votes
0
answers
744
views
Why is there no improvement when training Xgboost with pseudo-Huber loss?
In this StackOverflow post I asked if there was something wrong with my syntax when training an XGboost model (in R) with the native pseudo-Huber loss ...
- 1,021
2
votes
1
answer
816
views
How to calculate the running mean absolute deviation
I wish to calculate the running mean absolute deviation (MAD) without storing the previous n data points. This calculation is for a continuous stream of data, i.e. infinite length. I am trying to ...
- 121
0
votes
0
answers
361
views
For multivariate linear regression, what is the partial derivative for Mean Absolute Error?
What is the partial derivative for MAE for multivariate linear regression? I understand that for mean squared error (MSE) the partial derivative with respect to some $\theta_1$ would be $-\theta_1 \...
9
votes
1
answer
645
views
an upper bound of mean absolute difference?
Let $X$ be an integrable random variable with CDF $F$ and inverse CDF $F^*$. $Y$ is iid with $X$. Prove $$E|X-Y| \leq \frac{2}{\sqrt{3}}\sigma,$$ where $\sigma=\sqrt{Var(X)} = \sqrt{E[(X-\mu)^2]}$.
I ...
- 1,449
0
votes
1
answer
73
views
What are the drawbacks of aggregating regression results if error decreases?
I am trying to predict Total National Sales in dollars for a major retailer. However, the data I'm using exists at the state level, and includes several features around specific brands and foot ...
- 203
0
votes
0
answers
25
views
Why isn't absolute deviation used [duplicate]
If the reason that standard deviation is used is that it's easier to just square the distance every time than find the absolute value. Then why is √(E(x-m)^2)/n not used because if I'm not mistaken |x|...
- 1
1
vote
1
answer
38
views
Ho do I determine the probability of price reversals using linear regression?
In the screenshot, you will see the daily prices of Nasdaq. Each candle has a High, Low, Open and Close price.
I have drawn a regression line with a 2 standard deviation channel on either side.
How ...
- 113
0
votes
0
answers
18
views
We square at the variance and SD is it due a link with to Gaussian function? [duplicate]
I had almost same question as "Why square the difference instead of taking the absolute value in standard deviation?". But I also noticed the $\sigma$ or population standard deviation is ...
- 115
1
vote
1
answer
16
views
Alternative error measure--maximize data to have residuals less than given threshold
I am looking for a method which finds a linear model (2 input variables, everything reasonably close enough to normally distributed), but with an alternative measure of error. Every error measure I ...
- 129
5
votes
2
answers
114
views
Random variables $(X,Y)$ with $\text{Var}(X)<\text{Var}(Y)$ and $\mathbb{E}(|X-\mu_X|)>\mathbb{E}(|Y-\mu_Y|)$
I am looking for an example of a pair of random variables $(X,Y)$ with expected values $(\mu_X,\mu_Y)$ satisfying the following relationships:
$$
\text{Var}(X)<\text{Var}(Y)
$$ and
$$
\mathbb{E}(|X-...
- 61.1k
1
vote
0
answers
39
views
How to derive this MAE error bound on the central limit theorem?
Is this derived from Chebyshev's inequality or a tail bound theorem? If not, how was it derived?
Does this require the existence of the third moment?
Does this bound suggest the normal approximation ...
user271077
1
vote
0
answers
18
views
how to detect a 5% deviation between more than 2 numbers
Dear StackExchange Team
in the industry there are GAs generators that have hundreds of instrument measuring process values, such as pressure, temperatures, etc.
we have a simple case where a ...
- 11
0
votes
1
answer
788
views
How to fix heteroscedasticity (funnel shape)?
I am running a mlr in python on a dataset with 2D feature vectors, X1 and X2 on a single response, Y. The data ends up being funnel-shaped, as below:
X1 v Y, with the colors being X2.
It was ...
10
votes
2
answers
402
views
Is mean absolute deviation smaller than standard deviation for $n\ge 3$?
I want to compare the mean absolute deviation with standard deviation in general case with this definition:
$$MAD = \frac{1}{n-1}\sum_1^n|x_i - \mu|, \qquad SD = \sqrt{\frac{\sum_1^n(x_i-\mu)^2}{n-1}}...
- 201
0
votes
0
answers
69
views
Compare variables with zeta-score
Assume the calculation of a physical quantity $x$ from a given formula i.e. $$x=\dfrac{a*b}{c}$$ where $a, b$ and $c$ are experimental observables, therefore $x$ is a quantity derived from ...
- 121
1
vote
1
answer
532
views
Why do we use squared deviations to compute the SD, given that it amplifies the effect of outliers? [duplicate]
Suppose I have the following hypothetical data:
One thousand times value 15 (i.e., 15 occurs 1000 times)
and a single outlier value - 115 (i.e., 115 occurs just once - an outlier)
Thus the mean is: $...
- 119
0
votes
1
answer
524
views
Estimating Expected Order Statistics
I have a fairly basic question that I'm looking for a reference for.
First, a couple definitions. Let's say $X_1,\ldots,X_n$ are IID samples from a distribution $F$ over $[0,1]$. For any $k\in\{1,\...
- 1
1
vote
0
answers
387
views
Statistical test for comparing the performance of two methods using absolute differences
I am doing a study where I have to compare tumor sizes measured using two different methods with the real tumor size.
What I want to prove is that the absolute difference between method 1 and real ...
- 11
5
votes
2
answers
106
views
Rephrasing Statistics using MAD
I just got done reading Gorard's "Revisiting a 90-year-old debate: the advantages of the mean deviation." (https://www.leeds.ac.uk/educol/documents/00003759.htm) I'm no expert at statistics; in fact, ...
- 51
1
vote
2
answers
162
views
Verbal Description Of The RSE
I'm reading An Introduction to Statistical Learning, currently on page 69, the chapter on Simple Linear Regression.
The text says:
In the case of the advertising data, we see from the linear ...
- 113
-1
votes
2
answers
3k
views
Why does Mean deviation = Standard deviation = Range/2?
We know that mean deviation and standard deviation are two different things but why are both of them equal to Range/2?
Range ($R$) = Highest value - lowest value
Mean deviation ($\text{MD}$) = ...
- 11
6
votes
2
answers
4k
views
Why not use modulus for variance? [duplicate]
I am trying to wrap my mind around the variance definition.
Given a set of values S and n = #(S), the variance is defined as:
$$
\operatorname{var}(S) = \frac{\sum_{i=1}^n( S_i - \operatorname{mean}(...
1
vote
0
answers
186
views
An efficient algorithm for sliding mean deviation
I'm searching for a computationally efficient algorithm to calculate the sliding mean deviation from sample $x_a$ to sample $x_b$ belonging to a large set $x_0, x_1 ... x_n$ with the condition that $a$...
- 119
1
vote
1
answer
782
views
Best way to describe changes in performance through time.
I'm wondering how to explain changes in production/performance metrics quarter to quarter. In the example below, one can see that the average processing time for "widgets" has decreased substantially ...
0
votes
0
answers
33
views
Is standard deviation always larger than mean deviation? [duplicate]
If yes why is it the case that it have this property? There must be some mathematical proof or intuitive understanding in that case.
- 351
1
vote
1
answer
456
views
Why don't dispersions like median deviation and mode deviation exists on the lines of mean deviation?
Or to say in the formula MD=$\frac{1}{N}\sum_{i=1}^{N}|x_i-\overline{x}|$ why can't we have median or mode instead of mean($\overline{x}$) and as a result speak about additional possible measures of ...
- 351
2
votes
1
answer
1k
views
Is mean deviation the same as mean absolute difference?
Do both terms mean the same thing or are they different?
- 351
2
votes
1
answer
80
views
Variance of Binomial total deviation
I have a Binomial random variable $X \sim B(n,p)$. Is there a closed form expression or an upper bound for the variance of the absolute deviation $|X-\mathbb{E}[X]|$ ?
5
votes
1
answer
1k
views
Mean Absolute Deviation of Student t distributions?
I could only find these two formulae, both apparently with some issues.
Which is the correct one? Online references and derivation would be especially appreciated.
- 988
8
votes
0
answers
10k
views
Relative Mean Deviation [closed]
The Mean Deviation, also referred to as Absolute Average Deviation, is sometimes preferred to the Standard Deviation (Cf. Gorard 2004). For a sample, it can be expressed as:
$MD = \frac{1}{n} \sum\...
- 2,136
0
votes
0
answers
316
views
Difference between absolute deviation to population median and sample mean
I have independent variables $X_i\in[0;1]$ and suppose they are uniformly distributed. If you want to minimize the total absolute deviation to a fixed number, how much can you gain from using the ...
- 1,981