0
$\begingroup$

How can I tell if the majority of my data is either above or below my average?

8
8
7
5
4
9
8
9
9
11
13

AVG = 8.272727273
MEDIAN = 8
PERCENTILE 0.5 = 8

If my average is high compared to the rest of the data... then I will use percentile 0.4 as my cutoff so that I get a better representation of the data. But if my average is low compared to the rest of the data, then I will use a cutoff of percentile 0.6 to get a better representation of the data.

Is there a better way than comparing the median to the average? Should I be using something like a confidence interval instead or some kind of distribution?


EDIT:

At the end of my process I am running a pearson against the average, but in the scenario below there's a lot of good data to the right of the average so i would use the 0.6 instead of the average

enter image description here

$\endgroup$
7
  • $\begingroup$ You calculate the average (mean), count how many observations are below average, and divide that number by the total number of observations. I count 6 of 11 observations above the average in your data set. However, your comment about 0.4 as a cutoff makes me think that your real goal is something besides counting how many points are above the average, and you’re using that method to try to accomplish it. What is your goal? $\endgroup$
    – Dave
    Apr 16 '20 at 1:47
  • $\begingroup$ @Dave Thanks Dave. That makes sense! The following is a slight simplification. I have an aggregate metric (average OR percentile 0.4 OR percentile 0.6) about all the samples in my training set. Then, I am comparing that aggregate metric to a to a sample in my test set. Sometimes that comparison works better when I use the 0.4 and sometimes with the 0.6... so I am trying to determine how to code for that. $\endgroup$
    – Kalanos
    Apr 16 '20 at 2:02
  • $\begingroup$ I do not follow what you’re doing. It will be helpful to give details in an edit to your original question where you have more room to explain yourself than a comment allows. $\endgroup$
    – Dave
    Apr 16 '20 at 2:12
  • $\begingroup$ @Dave edited with a picture of a hypothetical distribution. $\endgroup$
    – Kalanos
    Apr 16 '20 at 2:50
  • $\begingroup$ Maybe skew is a better measure of what I am after. $\endgroup$
    – Kalanos
    Apr 16 '20 at 3:05
0
$\begingroup$

Edit: Please refer @Dave's comments.

Two options:

  1. One of the properties of normal distribution is that mean and median are same. You can simply check the normality of your data. If the p-value is <0.05, a variable is not normally distributed which means that the mean and median are not equal.

  2. Otherwise, you can perform a t.test for your data against the median, to check whether the mean of your data is significantly different from the expected mean(in this case, median).

$\endgroup$
1
  • 2
    $\begingroup$ There are problems with both of those. 1) First, $0.05$ isn’t some magic number. Second, we can get low p-values because of large sample sizes despite trivial deviations from the null hypothesis. Third, symmetric distribution can be highly non-normal, such as a Cauchy distribution. 2) This is a case of data snooping. You decide on your null and alternative hypotheses before you look at the data. Data snooping can result in a loss of control over error rates. $\endgroup$
    – Dave
    Apr 16 '20 at 2:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.