My team and I are working on a project that aims to measure the impact of adding more public bicycle stations in Mexico City on private vehicle traffic. We selected 100 key locations, which are spread around an area, and we have been collecting real-time data (from the Google Distance Matrix API) about how much time it would take to go from one location to another (all-vs-all combinations), at a rate of around 400 observations every 30 mins.

Since we have budget restrictions, we could not get a higher collection rate, and at first it seemed good enough. Later we realized that there are certain origin/destination combinations that are not useful for the analysis, and someone in the team suggested that we continued collecting random samples with a reduced universe of only the origin/destination locations which are useful so that we had more observations for them.

I'm having a hard time understanding if doing so would be problematic. So my question is: does reducing the universe of origin/destination combinations from which we are sampling invalidate the data we have collected up to that point? If we restrict all the analysis from the start, by "deleting" the observations for origin/destination combinations which we now know are not useful due to many reasons which are not "manipulating" or "trying to force results" in any way, is the difference in sampling universes an issue?

Any references would be appreciated.

  • 1
    $\begingroup$ This seems to hinge on why some origin/destination pairs are not "useful". Why aren't they useful? $\endgroup$ – RoryT Dec 9 '17 at 3:16
  • $\begingroup$ It was because bycicles are not allowed in some of the routes we were sampling from because of safety issues. However, in what case would it be harmful to do so? Thanks! $\endgroup$ – Omar Trejo Dec 11 '17 at 23:30
  • 2
    $\begingroup$ It sounds like these pairs are actually out of scope of your population, so it should be fine to exclude them UNLESS you are drawing conclusions about a population that includes routes which disallow bicycles. So make sure you adjust the scope of your study population and you should be okay. $\endgroup$ – RoryT Dec 14 '17 at 5:12

So, you discovered that some of the data you collected cannot be used, because you learnt that for some of the routes you sampled, bikes where not allowed. It would certainly not be meaningful to continue sampling useless data, so your proposal of restricting the sampling frame is the right thing to do, and cannot cause any problems.

Previous data is not invalidated in any sense.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.