# Does random sampling implies non-biased results?

Let's say that we want to estimate how much deforestation is done in the world considering 2 moments in time. We have a satellite that can take pictures anywhere in the world, but not of the entire world (or we're not willing to wait for the entire world to be pictured; I know there's a million better ways to do this, but let's ignore that for now).

Let's say that we take 10.000 pictures at random places on earth at moment A and 10 years later we take pictures of the same locations at moment B. Then someone manually goes through these 20.000 pictures and say deforestation/not-deforestation.

Obviously, there's going to be a bunch of pictures that were ocean at moment A, and were still ocean at moment B, which will be marked as not-deforestation.

Would the results of such a method be biased towards not-deforestation or just inaccurate because we did random sampling (which is supposed to always be unbiased) and we should have just increased the sampling size to increase accuracy?

My intuition tells me that the results would be biased because not-deforestation pictures would have a known higher chance of being selected when compared to deforested pictures, but I would like a more statistically sound argument, or perhaps to be shown why am I wrong.

Edit: I am not asking whether this would be a good method of detecting deforestation. Deforestation is just an example that suits the question because a large part of the sampling population (the entire world) is clearly not deforestation because it's water.

Let's consider the following definitions:

point A ---|--- point B ---|--- change class

forest -----|-- not forest -|--- deforestation

forest -----|-- forest ------|--- not deforestation

not forest-|-- not forest -|--- not deforestation

not forest-|-- forest ------|--- not deforestation

All the answers give a slightly different perspective on the matter, which helped me a lot to understand the problem. I will mark Tim's answer as the correct answer because he had the most contributions for my understanding, but I encourage everyone interested in this question to also read @Peter Flom's, @Lewiam's, and @bi_scholar's answers for a complete understanding.

• Why not no forest -> no forest = deforestation? Makes same sense as what you defined. Depending on choice between the two, you'd get over +/-70% "deforestation" (ocean), what suggests that such definition is problematic.
– Tim
Jul 1, 2019 at 12:42
• I don't care about the definition, because I am not really trying to figure out how to monitor deforestation. I also don't have my own satellite that took 10k pictures. If someone feels uncomfortable with using these definitions for deforestation, feel free to call forest = class A, not forest=class B, deforestation = change class C, and not deforestation = change class C . What I want to know is whether such method would be considered biased, or inaccurate. Jul 1, 2019 at 12:47
• If you don't care then why you care if your results are biased or not?
– Tim
Jul 1, 2019 at 12:48
• @Tim for learning purposes. I understand that the example is not ideal because it can stir discussions about definitions, but I just didn't want to give a class A, class B example, and I also couldn't come with the most suitable example Jul 1, 2019 at 12:51
• OK, I get it. I'll edit my answer accordingly.
– Tim
Jul 1, 2019 at 12:52

This has nothing to do with random sampling, but with using inaccurate definitions. You cannot count pictures containing ocean as "no deforestation", because, at least as I understand it, deforestation is the activity of removing forest, so if there was no forest, it couldn't be removed.

By random sampling, we mean sapling randomly from the population of interest. In your case, the population is areas that contained forest at time point A. You should gather images of such areas.

If you want to collect images of the whole globe hoping to catch areas that contain forest, then this would extremely inefficient. Quick googling suggests that approximately 8% of Earth's area are forests, so with such sampling you would need at least ten times more of the images. In such case, you would throw away over 90% of the data that did not contain any forest at time A.

Given your comment, that this is an abstract example for learning pourposes, I guess that what you're asking is if the fact that you are sampling the images from the whole Earth, while caring only about the small fraction of them, would it make the results biased "because not-deforestation pictures would have a known higher chance of being selected". If 8% of Earth's area are forests, then at time A you have 8% chance of sampling such areas, so at time A random sampling would not introduce any bias. At time B you would look at the same areas as at the time A, you would be looking if there was a change in the 8% of the 10 000 images (forest area at time A). The non-forest areas have highr chance of being sampled because they are more frequent, this suggests that there is no bias. The only problem is that your sample of the forest areas at time A is small, so this may influence how reliable is your estimate.

• Regardless of the quality and/or efficiency of the method. Would you classify the results as being biased and inaccurate, or unbiased and inaccurate? Jun 30, 2019 at 19:39
• Or simply wrong for a different reason, and is there a name for the error of sampling outside your population of interest? Jun 30, 2019 at 19:49
• @Andrei as I said, the procedure is wrong.
– Tim
Jun 30, 2019 at 20:00
• if we are to get technical, an area that was not forest cannot be deforested, and that's why it can be classified as not-deforested. Logically speaking, anyone would say an area that was ocean and is still ocean is not a deforested area. So there is no issue with the definition there. Jun 30, 2019 at 20:05
• @Andrei yes, you're correct
– Tim
Jul 1, 2019 at 16:41

Mainly to respond to the title question: No, random sampling does not imply unbiased results.

Generally in statistics "unbiased" (applied here to estimation; there are other uses of the term "unbiased" that don't fit here) means the following: The situation is that you want to estimate an unobserved population quantity or parameter, and you draw a sample. Then you compute an estimator as a function of the observations in the sample. Unbiased estimation means that the expected value of the estimator is equal to the population quantity you want to estimate. The expected value is computed from the sampling distribution of the estimator.

In fact there are several possible reasons for bias. 1) Bias may be caused by the sample. This is avoided by random sampling. 2) The chosen estimator may be biased for the quantity to estimate. 3) There may be issues with the taken measurements and the like. 2) and 3) can not be repaired by random sampling, so random sampling helps but cannot guarantee unbiased estimation.

The other answers seem to center around the question whether your estimator is an unbiased estimator of the (not directly observed) amount of deforestation in the population, addressing issue 2), maybe 3). This depends on how exactly deforestation is defined. I am not an expert on this and won't therefore comment on it, see the other answers. My answer is meant to clarify the framework and the general role of random sampling for unbiasedness.

• Thank you very much. This helps a lot. In my case, only if we consider the estimator to be biased, we can call the results biased. Otherwise, if we don't make judgements about the estimator (deforestation definition), and we assume it unbiased, then the results should be called unbiased. To put it shortly, the results are just as biased as the estimator. Jul 1, 2019 at 15:00

Estimates of what?

That is, you have to precisely define "deforestation" before you can tell what is biased or unbiased. Yes, random sampling will give unbiased results, but you have to define the population before you can figure out what to sample. So, is the population:

1. All points on the Earth?

2. All land on the Earth?

3. All forested land on Earth?

or something else?

• Thank you for answering. the population is the entire Earth, and please see my precise definition for deforestation in the Edit I made to the question. Jul 1, 2019 at 12:31

The biggest flaw in this method seems to be that the assumption that the rate of deforestation in one part of the earth is representative of the de-forestation of the whole population (earth).

I'm not an expert on deforestation, but I'm sure that de-forestation rates in Europe differ from the ones in Africa.

Ab better approach might be to observe the whole earth in form of pictures, sub-sample and hand-label them and then extrapolate.

Note that I have not addressed the definition of de-forestation, which may or may not be an issue on it's own.

• On top of different deforestation rates, we also know that the deforestation rate over oceans is 0. So the question stands whether the random sampling approach would be considered biased, or just inaccurate. Jul 1, 2019 at 12:33
• If you only have data from Europe and want to extrapolate to the de-forestation rate of the earth then yes, your results will be biased. This is not an issue with re-sampling. Your sample simply does not represent the population. Jul 1, 2019 at 12:42
• In my answer I am mentioning that I have my own personal satellite that takes pictures wherever I want on Earth. More specifically, it took 10k pictures randomly on earth at point A in time, and another 10k at the exact same locations at point B in time. Jul 1, 2019 at 12:44
• Okay, in that case you can use sampling techniques such as bootstrap to estimate the population mean and SE. Any bias introduced would be due to (1) your sample does not represent the population well enough or (2) your definition of de-forestation is flawed. Jul 1, 2019 at 12:49
• If you define ocean area as 'no deforestation', then this will obviously be reflected in the result. It's not a bias of the sampling procedure, it's simple how you define the statistic. If you feel that this gives a wrong picture on world-wide de-forestation then re-define the statistic as suggested by @Tim, if not, keep it. Jul 1, 2019 at 12:51

I'd like to add two further sources of bias:

• Living in a part of Europe where the forests slowly grow, but that also sees some shifting of the areas (over a decade 2002 - 2012, we saw ≈ 0.4 % of land area forest -> no forest and ≈ 0.8 % of land area no forest -> forest, my region has ≈ 42 % of land area being forest), I'd like to add that also original non-forest areas need to be surveyed in order to detect new forest growth. Otherwise, you'll be biased by the fact that you are systematically overlooking the possibility of new forests growing/existing forests expanding.

• Secondly (this is not clear to me from question and some comments). Sampling frequently involves several steps.

• in your case, that may be a satellite streaming image data along it's course, and then you sampling locations from that stream.
• in my case (analytical chemist), it's e.g. taking field samples and then reducing them into small lab samples suitable to put into some instrument.

In such situations,

• Unbiased sampling in a previous step may be impossible to repair later on.
• or (in "lucky" circumstances), repairing bias in a previous step may be possible but may require non-random sampling.
E.g. randomly sampling p % of the images of a satellite stream would be unbiased sampling of the statellite's images, but may yield a very biased sample wrt. the earth's surface depending on the satellite's orbit. To still arrive at unbiased sampling of the earth surface you may need a particular non-uniform sampling from the satellite images.

Last but not least, uniform random sampling wrt. the overall sampling procedure may be unbiased but it may not be an efficient sampling strategy (if efficiency is a concern).
I'd like to encourage you to look into literature about good sampling strategies for your application: A large tnumber of (2d) sampling strategies have been developed in particular for agricultural research together with guidelines in which cases which heuristics work well (and also wrt. to remote sensing). Developing a sampling strategy deserves substantial amounts of thought that takes into account the application/system under study.

There are different types of bias in research. Only one is sampling bias. With random sampling, you can minimize this kind of bias but it does not mean other types of biases are avoided. For example, if you sample randomly, you may minimize self-selection bias, but other biases such as those arising from a bad questionnaire, bad sample size, or bad data analysis methods may still exist.