3
$\begingroup$

I am using Seaborn to create a barplot.

sns.barplot(df['City Group'], df['revenue']) #uses default mean estimator

enter image description here

I am a bit confused about the error bars. What is the statistics to calculate these error bars and what these error bars actually represent? As per documentation, these error bars define the confidence interval and uncertainty around the estimate. In simple words, does it mean there is a 95% (default ci=95) chance that the mean will be falling in this range?

Can you explain in detail?

Also, when I change the value of ci='sd' (keeping the same estimator) as:`

sns.barplot(df['City Group'], df['revenue'], ci='sd')

enter image description here

The range of error bar increases for standard deviation. How this is calculated and what does it reflect?

Sample Data: (original source: Restaurant Revenue Prediction)

enter image description here

Improve this question
$\endgroup$
4
  • $\begingroup$ 95% of values fall between the black lines by default. Are you able to include your sample data? $\endgroup$ – David Erickson Dec 26 '20 at 5:13
  • $\begingroup$ please paste the data into your question instead of an image. I don't want to login to kaggle to download. Thank you. $\endgroup$ – David Erickson Dec 26 '20 at 11:08
  • $\begingroup$ @DavidErickson Sample Data Added. I don't think that it means 95% of values fall in this range. To verify I have used this Code black_line_big_cities = len ( df [ ( df ['City Group'] == 'Big Cities' ) & ( ( df['revenue'] > 4000000) & ( df ['revenue'] < 6000000) ) ] ) total_big_cities = len( df [ ( df ['City Group'] == 'Big Cities' ) ] ) black_line_big_cities / total_big_cities *100 Gives an output of 33.33 %. This means only 33% of values fall in this blackline range (as a rough estimation). Remember that y-axis values are defined by the estimator function which is a mean. $\endgroup$ – M.Zubair Akram Dec 26 '20 at 11:12
  • $\begingroup$ Yes, I think you are right. sorry, haven't done confidence intervals since college. Please see: statisticshowto.com/probability-and-statistics/… $\endgroup$ – David Erickson Dec 26 '20 at 11:31
1
$\begingroup$

By specifying ci='sd', you are indicating at what range you could be 95% sure that a sample of the population data contains the population standard deviation. From graphpad:

Interpreting the CI of the SD is straightforward. If you assume that your data were randomly and independently sampled from a Gaussian distribution, you can be 95% sure that the CI contains the true population SD.

Also, with only 137 rows of data, you are going to have a wider range for the confidence interval of your standard deviation than if you had let's say 1,000 + rows; because, generally, with a lower size n -- the higher the variance of the data.

Seaborn uses a technique to make inferences about population statistics using "Bootstrapping" per the documentation:

This is a basic concept of bootstrapping. "The basic idea of bootstrapping is that inference about a population from sample data (sample → population) can be modelled by resampling the sample data and performing inference about a sample from resampled data (resampled → sample). As the population is unknown, the true error in a sample statistic against its population value is unknown."

I hope this helps. I took stats in college and and understand the concept on a high-level, but I am not an expert in this space; so, I hope someone can offer some conclusions on how to interpret your results. If I were to make a conclusion (an extremely general one at that), you have a decent amount of variance in your data but also a small-medium sample size, which increases the range of your confidence interval. Finally, ci='95%' is a much more common way to analyze confidence rather than ci='sd' from some reading on the topic. For example, this article says "It's not done often, but it is certainly possible to compute a CI for a SD."

Improve this answer
$\endgroup$
1
  • $\begingroup$ thanks for the explanation. $\endgroup$ – M.Zubair Akram Dec 26 '20 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.