# Confidence intervals relating to non-parametric data

I have a question relating to computing confidence intervals that doesn't seem to be correct for my data. I have sales data on a test group and control group that looks something like this (note - the numbers have been made up just to show the kind of data I have):

Date        Group1 Control (Group1 - Control) Standardized(Group1 - Control)
X-Jan-20XX  123.5     187.2   (123.5-187.2)       -0.05
X           200.9     97.8
.            .         .          .                 .
.            .         .          .                 .
.            .         .          .                 .
Xn           Yn        Tn         Dn                Sn


I then compute the 95% confidence intervals on the standardized(group1 - control) using the standard method (shown below) to identify the line where we can safely assume the data should fall under.

The problem I have is shown through the image below, as you can see majority of the data falls outside of the confidence interval. It must be noted that my data is not normally distributed, however, when I standardize my data I get mean 0 and standard deviation of 1. Can someone help with solutions on this issue?

What I am suspicious of is the fact that my initial raw data is not normally distributed whilst the methods implemented are for normally distributed data. I believe this is why there are many data points that are not captured under the normal distribution. If this seems to be the problem, can someone suggest solutions?!?

Thanks!

• A mean of 0 and standard deviation of 1 is NOT what characterizes a normal distribution!! You can get any data set to have a mean of 0 and standard deviation of 1 by the z-score transform: $z_i=(x_i -\bar{x})/s$ for every data point $x_i$.
– Dave
Mar 2, 2020 at 11:45
• Thanks, that has been removed - do you have any comments on the rest of the question? Mar 2, 2020 at 11:50
• Welcome to the site! What do the gray points represent? Are you examining the difference in scores across two groups over time. Do you want to know why the difference in standardized scores lies outside of the confidence bands? Mar 2, 2020 at 12:23
• Thanks! The grey points are the plotted values of the column Standardized(group1 - control). Yes, I want to know why a majority of the standardized scores lie outside of the confidence intervals (the confidence intervals are generated based on the standardized data). I get the same results when I use raw values, the raw values are significantly outside the confidence interval. Mar 2, 2020 at 12:46
• Shouldn’t your best guess of the difference in population means be $\bar{X}_{1} - \bar{X}_{2}$? Just want to make sure you are calculating the correct margin of error. Mar 2, 2020 at 13:24

You mix up the standard-deviation of your data, and the standard-deviation of the mean of your data. The confidence interval you mention is for the mean of your data, and it is not an interval where you "can safely assume the data should fall under". It is - in loose tersms - the interval where you "can safely assume that the mean of your data falls under". While the mean of your data sample is a single fixed number, it is subject to uncertainty. This is because it is estimated from a finite amount of datapoints. The mean of the distribution that underlies the datapoints might in fact be (slightly) different. If you estimated the mean from a finite sample, and if your data is normally distributed, then you can compute the uncertainty of your mean via the formula $$\sigma/\sqrt{N}$$, and with this you can construct confidence intervals. This does however not mean that a certain fraction of your datapoints lie withing that confidence interval. If $$N$$ goes towards infinity, your confidence interval gets smaller and smaller - meaning that a smaller and smaller fraction of points lies within the confidence interval. The confidence interval tells you something else: if you redo your whole experiment (gather new data), and for each experiment compute the mean of your datapoints, how many of these means would lie within that confidence interval.