2
$\begingroup$

Backdrop - I will be doing stratified random sampling from a data that is about 100 million events. Distribution of this original data is extreme long tail (1% of objects contribute to 95% of events). I have 2 scenarios for sampling -

Scenario 1 - Doing a stratified random sample of size 100 each day for 30 days. Giving me 30 stratified samples. I combine to get 1 stratified sample of size 3000.

Scenario 2 - At the end of each month, I draw a single 3000 sized stratified sample.

Based on this I have 2 questions -

  1. Are the 2 approaches going to give me different distributions (relative to the original distribution)? Would they approximate to the original distribution?
  2. He I have arbitrarily taken 3000 sample size, What should the size of the sample be to represent the original population with say 90% confidence. Is there some equation I can use to calculate this?
  3. In approach-1; obviously after n days (1≤n≤30) as the cumulative data samples increases the confidence of representing the original 100 million population increases. Is there some way I can know - that after 5 days cumulative sample this data is say 20-25% confident that it represents population. This I want to calculate each day.

All things being same, I would prefer scenario-1 as it increases convenience for me.

Update: to the asked questions, providing more context -

  1. Each of these samples need to be labelled, hence can't work with entire 100 million events. Also there is a limitation of being able to label on X items per month. But I want to know the confidence of that X.
  2. These events are "downloads" & each object is "file" being downloaded. Am unsure about download patterns about each file. i.e. don't know if downloads of a file is more similar than downloads belonging to different files. How does this effect sampling? Whatever be the pattern (or bias) in the data I want to accurately capture it in my sample with some x% confidence.
  3. No the way the population of downloads are different (seasonal + random components).
$\endgroup$
  • $\begingroup$ What are the samples going to be used for? And why do you need samples; why can't you just work with the whole data set? $\endgroup$ – Creosote Oct 5 '15 at 19:24
  • $\begingroup$ Your proposed schemes all ignore "objects". Some questions: 1) What are these? 2 )Will events belonging to the same object be more similar than events belonging to different objects? 3) Can you identify the 1% of high volume objects in advance? $\endgroup$ – Steve Samuels Oct 6 '15 at 0:02
  • $\begingroup$ Another question: is the population of events identical on all days? $\endgroup$ – Steve Samuels Oct 6 '15 at 0:30
  • $\begingroup$ @SteveSamuels please find answers inline added to the question. $\endgroup$ – Srikar Appalaraju Oct 6 '15 at 3:44
  • $\begingroup$ @Creosote please find answers inline added to the question. $\endgroup$ – Srikar Appalaraju Oct 6 '15 at 3:44
1
$\begingroup$

The first approach, taking a new sample each day, is superior theoretically to taking a single sample if the distribution of X (eg. its mean) differs between days. By "superior", I mean 1) that estimated means, proportions, and other measures will be less biased than those in a single sample and 2) that standard errors will be smaller. To get these advantages, you must analyze the data according to the theory of stratified sampling: https://en.wikipedia.org/wiki/Stratified_sampling and http://www.stattrek.com/survey-research/stratified-sampling-analysis.aspx Or consult any sampling text. Ignoring this theory and simply pooling the data would lead to bias: each day has equal representation in the sample (n = 100), whereas the actual number of downloads per day will certainly vary.

Since you are focused on characteristics of downloads, there is no need to consider files in the design. Note that files of the sampled downloads are not representative of the population of files, since those with more downloads are overrepresented.

You stated that the convenience of the first design makes it attractive to you. Good sample design always takes into account cost, effort, and convenience.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.