We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [standard-deviation]

Standard deviation is the square root of the variance of a random variable, an estimator thereof, or a similar measure of the spread of a batch of data.

Filter by
Sorted by
Tagged with
413
votes
22answers
204k views

Why square the difference instead of taking the absolute value in standard deviation?

In the definition of standard deviation, why do we have to square the difference from the mean to get the mean (E) and take the square root back at the end? Can't we just simply take the absolute ...
57
votes
3answers
50k views

Why is sample standard deviation a biased estimator of $\sigma$?

According to the Wikipedia article on unbiased estimation of standard deviation the sample SD $$s = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (x_i - \overline{x})^2}$$ is a biased estimator of the SD of the ...
96
votes
4answers
99k views

Difference between standard error and standard deviation

I'm struggling to understand the difference between the standard error and the standard deviation. How are they different and why do you need to measure the standard error?
38
votes
5answers
371k views

What is the difference between a population and a sample?

What is the difference between a population and a sample? What common variables and statistics are used for each one, and how do those relate to each other?
27
votes
4answers
14k views

How to perform Student's t-test having only sample size, sample average and population average are known?

Student's $t$-test requires the sample standard deviation $s$. However, how do I compute for $s$ when only the sample size and sample average are known? For example, if sample size is $49$ and sample ...
24
votes
1answer
10k views

Standard deviation of binned observations

I have a dataset of sample observations, stored as counts within range bins. e.g.: min/max count 40/44 1 45/49 2 50/54 3 55/59 4 70/74 1 Now, ...
22
votes
5answers
19k views

Variance of a bounded random variable

Suppose that a random variable has a lower and an upper bound [0,1]. How to compute the variance of such a variable?
17
votes
1answer
6k views

Can we use leave one out mean and standard deviation to reveal the outliers?

Suppose I have normally distributed data. For each element of the data I want to check how many SDs it is away from the mean. There might be an outlier in the data (likely only one, but might be also ...
68
votes
4answers
231k views

How to 'sum' a standard deviation?

I have a monthly average for a value and a standard deviation corresponding to that average. I am now computing the annual average as the sum of monthly averages, how can I represent the standard ...
54
votes
3answers
28k views

Standard deviation of standard deviation

What is an estimator of standard deviation of standard deviation if normality of data can be assumed?
20
votes
5answers
4k views

Why are we using a biased and misleading standard deviation formula for $\sigma$ of a normal distribution?

It came as a bit of a shock to me the first time I did a normal distribution Monte Carlo simulation and discovered that the mean of $100$ standard deviations from $100$ samples, all having a sample ...
20
votes
1answer
11k views

What is the standard error of the sample standard deviation?

I read from there that the standard error of the sample variance is $$SE_{s^2} = \sqrt{\frac{2 \sigma^4}{N-1}}$$ What is the standard error of the sample standard deviation? I'd be tempted to guess ...
17
votes
3answers
11k views

Maximum value of coefficient of variation for bounded data set

In the discussion following a recent question about whether the standard deviation can exceed the mean, one question was raised briefly but never fully answered. So I am asking it here. Consider a ...
13
votes
3answers
8k views

Adding coefficients to obtain interaction effects - what to do with SEs?

I have a multivariate regression, which includes interactions. For example, to get the estimate of the treatment effect for the poorest quintile I need to add the coefficients from the treatment ...
32
votes
3answers
100k views

Is it possible to find the combined standard deviation?

Suppose I have 2 sets: Set A: number of items $n= 10$, $\mu = 2.4$ , $\sigma = 0.8$ Set B: number of items $n= 5$, $\mu = 2$, $\sigma = 1.2$ I can find the combined mean ($\mu$) easily, but how am ...
29
votes
5answers
152k views

How do I calculate a weighted standard deviation? In Excel?

So, I have a data set of percentages like so: ...
7
votes
2answers
7k views

Normality of residuals vs sample data; what about t-tests?

An addition to the common confusion about normality testing in inferential statistics for general linear models: I understand the assumption of normality refers to the residuals in ANOVA and ...
4
votes
2answers
128k views

What does the size of the standard deviation mean? [duplicate]

According to Wikipedia (my emphasis), In statistics, the standard deviation ... is a measure that is used to quantify the amount of variation or dispersion of a set of data values. A standard ...
128
votes
6answers
718k views

What's the difference between variance and standard deviation?

I was wondering what the difference between the variance and the standard deviation is. If you calculate the two values, it is clear that you get the standard deviation out of the variance, but what ...
11
votes
3answers
4k views

How can I find the standard deviation of the sample standard deviation from a normal distribution?

Forgive me if I've missed something rather obvious. I'm a physicist with what is essentially a (histogram) distribution centered about a mean value that approximates to a Normal distribution. The ...
14
votes
3answers
1k views

Why does this excerpt say that unbiased estimation of standard deviation usually isn't relevant?

I was reading on the computation of the unbiased estimation of standard deviation and the source I read stated (...) except in some important situations, the task has little relevance to ...
83
votes
10answers
59k views

Understanding “variance” intuitively

What is the cleanest, easiest way to explain someone the concept of variance? What does it intuitively mean? If one is to explain this to their child how would one go about it? It's a concept that I ...
12
votes
1answer
31k views

Transform Data to Desired Mean and Standard Deviation

I am looking for a method to transform my dataset from its current mean and standard deviation to a target mean and a target standard deviation. Basically, I want to shrink/expand the dispersion and ...
10
votes
3answers
12k views

Calculating distribution from min, mean, and max

Suppose I have the minimum, mean, and maximum of some data set, say, 10, 20, and 25. Is there a way to: create a distribution from these data, and know what percentage of the population likely lies ...
4
votes
1answer
1k views

lognormal distribution, standard-deviation and (physical) units

I have a distribution of microparticles that follows a lognormal distribution. The cumulative distribution function thus is given by: $$ F_X(x;\mu,\sigma) = \frac12 \operatorname{erfc}\!\left(-\frac{\...
31
votes
8answers
15k views

What is a standard deviation?

What is a standard deviation, how is it calculated and what is its use in statistics?
26
votes
1answer
73k views

Converting standard error to standard deviation?

Is it sensible to convert standard error to standard deviation? And if so, is this formula appropriate? $$SE = \frac{SD}{\sqrt{N}}$$
19
votes
4answers
31k views

Can mean plus one standard deviation exceed maximum value?

I have mean 74.10 and standard deviation 33.44 for a sample that has minimum 0 and maximum 94.33. My professor asks me how can mean plus one standard deviation exceed the maximum. I showed her ...
6
votes
2answers
12k views

Error bars, linear regression and “standard deviation” for point

I have a set of experimental data points. I performed the measurements in triplicate, for each of the point of the data set. Therefore, I can draw each data point with the standard deviation of each ...
18
votes
5answers
14k views

2D analog of standard deviation?

Consider the following experiment: a group of people is given a list of cities, and asked to mark the corresponding locations on an (otherwise unlabeled) map of the world. For each city, you will get ...
4
votes
1answer
718 views

How does one go about determining the standard deviation of an entire sample data set?

I have frequently been faced with analyzing csv files that are limited in length to approximately 32,000 records in length (overflows at 32,768, from signed integer limits on a 16 bit wide data bus). ...
34
votes
11answers
106k views

Mean absolute deviation vs. standard deviation

In the text book "New Comprehensive Mathematics for O Level" by Greer (1983), I see averaged deviation calculated like this: Sum up absolute differences between single values and the mean. Then ...
32
votes
6answers
3k views

What would a robust Bayesian model for estimating the scale of a roughly normal distribution be?

There exists a number of robust estimators of scale. A notable example is the median absolute deviation which relates to the standard deviation as $\sigma = \mathrm{MAD}\cdot1.4826$. In a Bayesian ...
15
votes
9answers
22k views

Can the standard deviation of non-negative data exceed the mean?

I have some triangulated 3D meshes. The statistics for the triangle areas are: Min 0.000 Max 2341.141 Mean 56.317 Std dev 98.720 So, does it mean anything particularly useful about the standard ...
9
votes
2answers
8k views

When using SVMs, why do I need to scale the features?

According to the documentation of the StandardScaler object in scikit-learn: For instance many elements used in the objective function of a learning algorithm (such as the RBF kernel of Support ...
8
votes
2answers
2k views

Confidence interval for the median

I have a set of values ${x_i}, i=1, \dots ,N$ of which I calculate the median M. I was wondering how I could calculate the error on this estimation. On the net I found that it can be calculated as $1....
6
votes
2answers
34k views

Standard deviation of a ratio (percentage change)

I have 2 data sets. The first data set, let's call it $X$ has an average value of ($\bar X$) and standard deviation of ($STD_X$), the second set of data also has the average value of ($\bar Y$) and ...
4
votes
7answers
21k views

How to judge if 5 point Likert scale data are normally distributed?

I have read that the t-test is used when the population is normally distributed. How can I determine if my data are normal given that I am using 5-point Likert scale with a sample size of 100? What ...
7
votes
2answers
2k views

Average and standard deviation of timestamps (time wraps around at midnight)

I have lots of sensor data with timestamps like "2014-09-09 16:10:45" and accompanying sensor readings. To get some insight into these I want to find "unusual" events by looking at the average and ...
6
votes
2answers
483 views

How to express “inequality” of a distribution in one number?

I've got an educational game in the work that challenges players to redraw U.S. state borders in a way that reduce the inequality between big-population and small-population states. As they play, they ...
5
votes
4answers
6k views

z-score and Normal Distribution

It's my understanding that a z-score can only be calculated and accurately used for data sets that are normally distributed. Since "perfect" normal distribution almost never occurs in real-world data ...
4
votes
2answers
6k views

Population Variance and Sample Variance

How is it that lowering the number of frequence,$n$ by $1$ in the for formula for Population and Sample Variance account for the discrepency of using sample rather than population. I mean how is ...
7
votes
1answer
11k views

Understanding the calculation of the independent samples Cohen's d from t & df

It seems that for calculation of Cohen's $d$ (an estimate of effect size), you should use the difference in means (or in one-sample case, from baseline), divided by $SD$, if that information is ...
6
votes
1answer
677 views

How can the confidence interval for standard deviation not include the sample standard deviation?

I'm using the standard formula for the confidence interval for standard deviation: SQRT(((N-1)*s^2)/(CHISQ.INV((1-Confidence)/2, N-1))) By mistake I entered a ...
5
votes
1answer
30k views

How to interpret two standard deviations below the mean of a count variable being less than zero?

I asked people how many times they visited their local pub in a 'normal' week. The result can be zero, one, two, three, four, and five and more. The mean is 2 and the standard deviation is 1.3. So ...
8
votes
1answer
22k views

RMSE vs Standard deviation in population

RMSE (Root mean square error) and SD (Standard deviation) have similar formulas. This link says The only difference is that you divide by $n$ and not $n−1$ since you are not subtracting the ...
16
votes
2answers
17k views

Why is the standard deviation defined as sqrt of the variance and not as the sqrt of sum of squares over N?

Today I taught an introductory class of statistics and a student came up to me with a question, which I rephrase here as: "Why is the standard deviation defined as sqrt of variance and not as the sqrt ...
25
votes
4answers
8k views

Intuition behind standard deviation

I'm trying to gain a better intuitive understanding of standard deviation. From what I understand it is representative of the average of the differences of a set of observations in a data set from ...
14
votes
5answers
5k views

Is there a measure of 'evenness' of spread?

I looked up on the web, but couldn't find anything helpful. I'm basically looking for a way to measure how 'evenly' a value is distributed. As in, an 'evenly' distributed distribution like X: and an ...
16
votes
2answers
737 views

For which distributions is there a closed-form unbiased estimator for the standard deviation?

For the normal distribution, there is an unbiased estimator of the standard deviation given by: $$\hat{\sigma}_\text{unbiased} = \frac{\Gamma(\frac{n-1}{2})}{\Gamma(\frac{n}{2})} \sqrt{\frac{1}{2}\...