Questions tagged [standard-deviation]

Standard deviation is the square root of the variance of a random variable, an estimator thereof, or a similar measure of the spread of a batch of data.

Filter by
Sorted by
Tagged with
399
votes
21answers
195k views

Why square the difference instead of taking the absolute value in standard deviation?

In the definition of standard deviation, why do we have to square the difference from the mean to get the mean (E) and take the square root back at the end? Can't we just simply take the absolute ...
124
votes
6answers
683k views

What's the difference between variance and standard deviation?

I was wondering what the difference between the variance and the standard deviation is. If you calculate the two values, it is clear that you get the standard deviation out of the variance, but what ...
95
votes
4answers
97k views

Difference between standard error and standard deviation

I'm struggling to understand the difference between the standard error and the standard deviation. How are they different and why do you need to measure the standard error?
79
votes
10answers
57k views

Understanding “variance” intuitively

What is the cleanest, easiest way to explain someone the concept of variance? What does it intuitively mean? If one is to explain this to their child how would one go about it? It's a concept that I ...
64
votes
4answers
220k views

How to 'sum' a standard deviation?

I have a monthly average for a value and a standard deviation corresponding to that average. I am now computing the annual average as the sum of monthly averages, how can I represent the standard ...
54
votes
3answers
47k views

Why is sample standard deviation a biased estimator of $\sigma$?

According to the Wikipedia article on unbiased estimation of standard deviation the sample SD $$s = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (x_i - \overline{x})^2}$$ is a biased estimator of the SD of the ...
51
votes
4answers
27k views

Standard deviation of standard deviation

What is an estimator of standard deviation of standard deviation if normality of data can be assumed?
39
votes
5answers
366k views

What is the difference between a population and a sample?

What is the difference between a population and a sample? What common variables and statistics are used for each one, and how do those relate to each other?
39
votes
3answers
65k views

What does standard deviation tell us in non-normal distribution

In a normal distribution, the 68-95-99.7 rule imparts standard deviation a lot of meaning, but what would standard deviation mean in a non-normal distribution (multimodal or skewed)? Would all data ...
33
votes
10answers
99k views

Mean absolute deviation vs. standard deviation

In the text book "New Comprehensive Mathematics for O Level" by Greer (1983), I see averaged deviation calculated like this: Sum up absolute differences between single values and the mean. Then ...
32
votes
3answers
95k views

Is it possible to find the combined standard deviation?

Suppose I have 2 sets: Set A: number of items $n= 10$, $\mu = 2.4$ , $\sigma = 0.8$ Set B: number of items $n= 5$, $\mu = 2$, $\sigma = 1.2$ I can find the combined mean ($\mu$) easily, but how am ...
31
votes
8answers
14k views

What is a standard deviation?

What is a standard deviation, how is it calculated and what is its use in statistics?
31
votes
6answers
3k views

What would a robust Bayesian model for estimating the scale of a roughly normal distribution be?

There exists a number of robust estimators of scale. A notable example is the median absolute deviation which relates to the standard deviation as $\sigma = \mathrm{MAD}\cdot1.4826$. In a Bayesian ...
30
votes
4answers
20k views

What are the relative merits of Winsorizing vs. Trimming data?

Winsorizing data means to replace the extreme values of a data set with a certain percentile value from each end, while Trimming or Truncating involves removing those extreme values. I always see ...
28
votes
5answers
143k views

How do I calculate a weighted standard deviation? In Excel?

So, I have a data set of percentages like so: ...
26
votes
5answers
11k views

Why do we take the square root of variance to create standard deviation?

Sorry if this is has been answered elsewhere, I haven't been able to find it. I am wondering why we take the square root, in particular, of variance to create the standard deviation? What is it about ...
26
votes
1answer
71k views

Converting standard error to standard deviation?

Is it sensible to convert standard error to standard deviation? And if so, is this formula appropriate? $$SE = \frac{SD}{\sqrt{N}}$$
25
votes
4answers
14k views

How to perform Student's t-test having only sample size, sample average and population average are known?

Student's $t$-test requires the sample standard deviation $s$. However, how do I compute for $s$ when only the sample size and sample average are known? For example, if sample size is $49$ and sample ...
24
votes
4answers
7k views

Intuition behind standard deviation

I'm trying to gain a better intuitive understanding of standard deviation. From what I understand it is representative of the average of the differences of a set of observations in a data set from ...
23
votes
1answer
10k views

Standard deviation of binned observations

I have a dataset of sample observations, stored as counts within range bins. e.g.: min/max count 40/44 1 45/49 2 50/54 3 55/59 4 70/74 1 Now, ...
22
votes
5answers
17k views

Variance of a bounded random variable

Suppose that a random variable has a lower and an upper bound [0,1]. How to compute the variance of such a variable?
20
votes
5answers
4k views

Why are we using a biased and misleading standard deviation formula for $\sigma$ of a normal distribution?

It came as a bit of a shock to me the first time I did a normal distribution Monte Carlo simulation and discovered that the mean of $100$ standard deviations from $100$ samples, all having a sample ...
19
votes
2answers
69k views

What formula is used for standard deviation in R?

What formula is used in the standard deviation function sd in R?
19
votes
4answers
30k views

Can mean plus one standard deviation exceed maximum value?

I have mean 74.10 and standard deviation 33.44 for a sample that has minimum 0 and maximum 94.33. My professor asks me how can mean plus one standard deviation exceed the maximum. I showed her ...
18
votes
4answers
2k views

Is variance a more fundamental concept than standard deviation?

On this psychometrics website I read that [A]t a deep level variance is a more fundamental concept than the standard deviation. The site doesn't really explain further why variance is meant to ...
17
votes
3answers
10k views

Maximum value of coefficient of variation for bounded data set

In the discussion following a recent question about whether the standard deviation can exceed the mean, one question was raised briefly but never fully answered. So I am asking it here. Consider a ...
17
votes
3answers
39k views

How to interpret root mean squared error (RMSE) vs standard deviation?

Let's say I have a model that gives me projected values. I calculate RMSE of those values. And then the standard deviation of the actual values. Does it make any sense to compare those two values ...
17
votes
1answer
6k views

Can we use leave one out mean and standard deviation to reveal the outliers?

Suppose I have normally distributed data. For each element of the data I want to check how many SDs it is away from the mean. There might be an outlier in the data (likely only one, but might be also ...
17
votes
1answer
9k views

What is the standard error of the sample standard deviation?

I read from there that the standard error of the sample variance is $$SE_{s^2} = \sqrt{\frac{2 \sigma^4}{N-1}}$$ What is the standard error of the sample standard deviation? I'd be tempted to guess ...
17
votes
5answers
13k views

2D analog of standard deviation?

Consider the following experiment: a group of people is given a list of cities, and asked to mark the corresponding locations on an (otherwise unlabeled) map of the world. For each city, you will get ...
16
votes
5answers
4k views

Comparing the variance of paired observations

I have $N$ paired observations ($X_i$, $Y_i$) drawn from a common unknown distribution, which has finite first and second moments, and is symmetric around the mean. Let $\sigma_X$ the standard ...
16
votes
2answers
708 views

For which distributions is there a closed-form unbiased estimator for the standard deviation?

For the normal distribution, there is an unbiased estimator of the standard deviation given by: $$\hat{\sigma}_\text{unbiased} = \frac{\Gamma(\frac{n-1}{2})}{\Gamma(\frac{n}{2})} \sqrt{\frac{1}{2}\...
16
votes
2answers
17k views

Why is the standard deviation defined as sqrt of the variance and not as the sqrt of sum of squares over N?

Today I taught an introductory class of statistics and a student came up to me with a question, which I rephrase here as: "Why is the standard deviation defined as sqrt of variance and not as the sqrt ...
16
votes
2answers
2k views

Why is it called the “standard” deviation?

I have a simple - and possibly obviously trivial - question: why is the standard deviation called just that, "standard"? Is it because it standardizes the comparison of data sets and results with ...
15
votes
5answers
803 views

Why do US and UK Schools Teach Different methods of Calculating the Standard Deviation?

As I understand UK Schools teach that the Standard Deviation is found using: whereas US Schools teach: (at a basic level anyway). This has caused a number of my students problems in the past as ...
15
votes
9answers
21k views

Can the standard deviation of non-negative data exceed the mean?

I have some triangulated 3D meshes. The statistics for the triangle areas are: Min 0.000 Max 2341.141 Mean 56.317 Std dev 98.720 So, does it mean anything particularly useful about the standard ...
15
votes
3answers
172k views

How do I evaluate standard deviation?

I have collected responses from 85 people on their ability to undertake certain tasks. The responses are on a five point Likert scale: 5 = Very Good, 4 = Good, 3 = Average, 2 = Poor, 1 = Very Poor, ...
15
votes
4answers
34k views

What is the correlation if the standard deviation of one variable is 0?

As I understand, we can get correlation by normalizing covariance using the equation $$\rho_{i,j}=\frac{cov(X_i, X_j)}{\sigma_i \sigma_j}$$ where $\sigma_i=\sqrt{E[(X_i-\mu_i)^2]}$ is the standard ...
15
votes
2answers
4k views

Median absolute deviation (MAD) and SD of different distributions

For normally distributed data, the standard deviation $\sigma$ and the median absolute deviation $\text{MAD}$ are related by: $\sigma=\Phi^{-1}(3/4)\cdot \text{MAD}\approx1.4826\cdot\text{MAD},$ ...
15
votes
3answers
11k views

Computation of new standard deviation using old standard deviation after change in dataset

I have an array of $n$ real values, which has mean $\mu_{old}$ and standard deviation $\sigma_{old}$. If an element of the array $x_i$ is replaced by another element $x_j$, then new mean will be $\...
14
votes
3answers
67k views

Why does the standard deviation not decrease when I do more measurements? [duplicate]

I made 100 measurements of a certain quantity, calculated mean and standard deviation (with MySQL), and got mean=0.58, SD=0.34. The std seemed too high relative to the mean, so I made 1000 ...
14
votes
3answers
1k views

Why does this excerpt say that unbiased estimation of standard deviation usually isn't relevant?

I was reading on the computation of the unbiased estimation of standard deviation and the source I read stated (...) except in some important situations, the task has little relevance to ...
14
votes
3answers
10k views

When would it be appropriate to report variance instead of standard deviation?

I have conducted an analysis in which I have modeled different variance components. When reporting the results in a table, it is much more concise to report standard deviations instead of variances. ...
14
votes
5answers
4k views

Is there a measure of 'evenness' of spread?

I looked up on the web, but couldn't find anything helpful. I'm basically looking for a way to measure how 'evenly' a value is distributed. As in, an 'evenly' distributed distribution like X: and an ...
14
votes
2answers
9k views

Relationship between the range and the standard deviation

In an article I found the formula for the standard deviation of a sample size $N$ $\sigma=\frac{\overline{R}}{2.534}$ where $\overline{R}$ is the average range of subsamples (size $6$) from the ...
14
votes
2answers
1k views

Regression to the mean in “Thinking, Fast and Slow”

In Thinking, Fast and Slow, Daniel Kahneman poses the following hypothetical question: (P. 186) Julie is currently a senior in a state university. She read fluently when she was four years old. ...
13
votes
5answers
892 views

Bias towards natural numbers in the case of least squares

Why do we seek to minimize x^2 instead of minimizing |x|^1.95 or |x|^2.05. Are there reasons ...
13
votes
3answers
8k views

Adding coefficients to obtain interaction effects - what to do with SEs?

I have a multivariate regression, which includes interactions. For example, to get the estimate of the treatment effect for the poorest quintile I need to add the coefficients from the treatment ...
13
votes
3answers
4k views

Standard deviation of several measurements with uncertainties

I have two 2 hours of GPS data with a sampling rate of 1 Hz (7200 measurements). The data are given in the form $(X, X_\sigma, Y, Y_\sigma, Z, Z_\sigma)$, where $N_\sigma$ is the measurement ...
12
votes
11answers
6k views

Is standard deviation totally wrong? How can you calculate std for heights, counts and etc (positive numbers)?

Let's say I am calculating heights (in cm) and the numbers must be higher than zero. Here is the sample list: ...