I have a sample of 2000 projects randomly extracted from a population of 10,000,000 following an exponential distribution. From this sample I would like to check if a statistical dependency exists between the number of lines of code and the number of users.

Instead of comparing these two features with Spearman's rank correlation directly on the 2000 projects, I've grouped the projects in the sample according to the number of users (obtaining 10 groups), and for each group I've calculated the average of lines of code.

The corresponding rho and p-values show that there is a significant dependency between such variables. On the contrary, if I run the Spearman's rank correlation on the 2000 projects, this dependency does not appear.

Can I say that:

  1. the 2000 projects are representative enough with respect to the population?
  2. there exists a dependency between number of users and lines of code, only if the number of lines of code for a given project is above the average?

Quite different questions are bundled together here.

  1. As you say you have selected a sample of 2000 randomly, that should impart good properties to the data you analyse, assuming that "random" here really does imply selection using random numbers or some equivalent device, not an arbitrary or haphazard selection. Note that random samples are not guaranteed to be representative of the population in the sense that a random sample faithfully reproduces all the properties of the population. In a strict sense, that is impossible. In any looser sense, everything depends on what you understand by "representative". Contrary to widespread belief, "representative of the population" does not have a precise statistical meaning except in the limiting and tautological sense that the population is representative of the population. See the definitive series of papers by W.H. Kruskal and F. Mosteller starting with William H. Kruskal and Frederick Mosteller. 1979. Representative sampling, I: Non-scientific literature. International Statistical Review 47: 13–24.

  2. Averaging over groups tends to reduce noise and often produces higher correlations. Your use of Spearman's correlation rather than Pearson's correlation makes that effect a little indirect. In general, correlations between values for groups and correlations between values for individuals often differ for this and other reasons. This has been studied under several headings, most bitingly in terms of Simpson's paradox, perhaps better called an amalgamation paradox, in view of its earlier discovery by Pearson and Yule at least. Broadly, it is hazardous to extrapolate both from group properties to individuals and vice versa.

  3. You introduce in your second question the condition "only if the number of lines of code for a given project is above the average", but nothing in your earlier text points to selection of values above such an average. Your explanation refers only to averaging by groups according to number of users, which sounds a quite different procedure.

  • $\begingroup$ Thank you for answering.Regarding your first answer,I meant with "representative" that the sample should follow the same distribution of the population for the two variables. If I know the size of the population and the corresponding distribution, I could fix the confidence level (95%) and the confidence interval (1.96) to calculate the minimum sample size? Regarding your second answer, I used Spearman's correlation, since I wanted to look for a monotonic relationship. Finally, concerning your third answer, I thought I could use the avg value to say something more about correlation identified. $\endgroup$ – user2314405 Mar 29 '14 at 13:13
  • $\begingroup$ Can I consider the correlation between the numbers of users and the average of lines of code in a project as a meaningful result? thanks. $\endgroup$ – user2314405 Mar 29 '14 at 13:18
  • $\begingroup$ If you can't find a strong correlation by using project-level data, the relationship appears too weak to take very seriously. You might get a better answer by showing us (a graph of) the data. $\endgroup$ – Nick Cox Mar 29 '14 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.