3
$\begingroup$

I am having a conceptual problem, of not being able to grasp whether my data is paired or unpaired. I have to test for the mean and median differences between two sets of data: SBP and GICS (see picture). The sample population is 1,500 firms for all years from 2004 to 2018. The firms are not necessarily the same, i.e. the 1,500 firms in 2005 are slightly different ones in 2006. From this population, both the SBP and GICS creates groupings of 10 from the 1,500 firms - but do it differently.

After the SBP and GICS have created the groupings, they take an average of the group - which becomes the variable/observation for each system (technically, this is an absolute percentage error). The goal is to look at which grouping system generates the lowest absolute percentage error. I have tried to illustrate it with the following:

enter image description here

My initial thought was that the SBP and GICS were unpaired; i.e. there was no "before vs. after" medical treatment effect, since both "systems" does the exact same thing: to create groupings of 10. What matters is, how they select the 10 firms for each group. Can someone help?

I was left wondering this, when I tried to run the tests in R. The sample median and mean values of the SBP and GICS are the following:

enter image description here

If we take the "simple" difference of the median, it is clearly 1.5% and 2.2% for the simple mean. My understanding is, that when I calculate the statistical pairwise difference (unreported from the figure), this value should be LOWER than the "simple" difference. Running the wilcox.test in R, with the following code:

wilcox.test(GICS, SBP, alternative = "two.sided", mu=0, conf.int=T, conf.level = 0.99, paired = TRUE

gives me a pairwise difference of 1.9%, where I am left wondering, why I can be in a situation where the pseudo median (population estimator) of a Wilcoxon signed rank test, can be larger than the simple differences between sample medians? I am performing a paired signed rank test, since my data is dependent.

I have uploaded my dataset here: Dataset as a .csv file.

Can someone help my understanding?

My main questions summarized:

  • (1) Is my sample paired or unpaired?
  • (2) By running the Wilcoxon signed rank test and Student t-test, can the generated pseudo median (or mean) EVER be larger, than the simple difference between sample medians and means?

EDIT: Running the Wilcoxon test as paired, i.e. pair=TRUE, gives me a pseudo median of 1.513%.

Improve this question
$\endgroup$
4
  • $\begingroup$ The value and code you give for the wilcox.test appear to be for the unpaired Wilcoxon test. Please can you give more details about what happens when you run the paired Wilcoxon test with 'paired=TRUE'? $\endgroup$
    – Izy
    Commented May 7, 2019 at 11:52
  • $\begingroup$ I edited the post. However, I can see that an issue arises with the example I've chosen, which doesn't reflect my concern. I have edited the post, so it is more clear. $\endgroup$
    – Philip
    Commented May 7, 2019 at 11:59
  • 1
    $\begingroup$ I think your question 2. has now been answered by @Glen_b in another one of your questions - as a 'yes'? stats.stackexchange.com/questions/406984/… $\endgroup$
    – Izy
    Commented May 7, 2019 at 14:01
  • $\begingroup$ I think so too. $\endgroup$
    – Glen_b
    Commented May 11, 2019 at 2:02

1 Answer 1

Reset to default
1
$\begingroup$

  1. My understanding of what you are doing is as follows:

    • You have a dataset for each year.
    • You have two systems, SBP and GICS, that each use the same input dataset (each year), but follow different processes so each come up with a different grouping. (I am assuming here that they each just create one grouping per year. It's not clear to me from your question if it's actually multiple groupings).
    • Assuming we're just talking about a single grouping per year, you then calculate a value based on that grouping, for each system.
    • You want to compare the values calculated by each system.

Based on this, I would say that you do have paired data, and that the pairs are by year (2006, 2007 etc.).

If you are actually comparing the outcomes of multiple groupings, over multiple years, then that would make things more complicated.

Also, an assumption of both the Student's t-test and Wilcoxon test is that your data points are independent. If this assumption is violated, then you cannot rely on the results. If you have the same firms in your dataset in different years, then I would argue that your data points are not in fact fully independent, and you should address this first before carrying out your test.

  1. The Wilcoxon test is actually a test of ranks, not medians. I see that you have a separate question on the pseudomedian, which I haven't come across before. From what I understand from the statement 'The pseudomedian of a distribution F is the median of the distribution of (u+v)/2, where u and v are independent, each with distribution F. If F is symmetric, then the pseudomedian and median coincide.' from ?wilcox.test as provided by Łukasz Deryło, then it stands to reason that if F is not symmetric, the pseudomedian and median will not be the same. I can't see any reason why the pseudomedian should be more likely to be 'wrong' in one direction (smaller) than the other (larger), but someone who understands it more might have an insight into this.
Improve this answer
$\endgroup$
8
  • $\begingroup$ Hi Izy, thank for you taking the time. You almost got it right; we actually do create amount of groupings EQUAL to the number of firms per year, 1500. The SBP and GICS simply creates 1500 different groups, since they are two "systems" for detecting similar companies. In other words, they do create multiple groupings. I did rerad that comment, which comes from the R help page, but it didn't really help me understand the concept of a different median. $\endgroup$
    – Philip
    Commented May 7, 2019 at 12:22
  • $\begingroup$ So you have 1500 firms per year, 10 firms per group, but 1500 different groups per year? So at least some firms must be included in more than one group? It's not clear to me what values you are then going on to test, but I think that you need to be very careful about ensuring that you have addressed the point about independence of observations before proceeding. $\endgroup$
    – Izy
    Commented May 7, 2019 at 12:30
  • $\begingroup$ There are 1500 firms in total, for every year. Both systems, SBP and GICS, creates a group of 10 companies, whose average value is computed and applied to a SINGLE firm. Therefore, both systems end up producing 1500 values (absolute percentage errors) per year. Hope that helps $\endgroup$
    – Philip
    Commented May 7, 2019 at 12:34
  • $\begingroup$ I'm not clear on what your exact research question is - perhaps you're interested in how representative the 10 firms chosen by each system are of the specific dataset presented to the system. In which case there are other things you might want to consider - do firms for which there is a large error differ by SBP/GICS and is there something systematically similar about those firms (i.e. are the systems biased in some way?). If you're more interested in how the systems would perform if making a prediction for a new dataset, then splitting your data into training and test datasets seems sensible. $\endgroup$
    – Izy
    Commented May 7, 2019 at 13:38
  • 1
    $\begingroup$ "I can't see any reason why the pseudomedian should be more likely to be 'wrong' in one direction (smaller) than the other (larger)" -- if you can generate a difference in one direction you can do it in the opposite direction by flipping the whole example around some point. $\endgroup$
    – Glen_b
    Commented May 11, 2019 at 2:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.