2
$\begingroup$

The scenario: Suppose there is a square shaped region with three cities in it. A dataset is collected by citizens, who collect data where they live. Thus, data is collected much more heavily in those three parts of the state.

Traditional response I see in the literature: This uneven distribution creates a spatial bias. Overlay a grid across the state, and sample (say) 100 observations from each grid cell, assuming each cell has 100 datapoints in it. This way, each part of the state is evenly represented in the data. This may result in throwing out lots of data in some cells (the ones near or in cities), but that is okay and better than having the bias.

Me: I do not understand why having unevenly distributed data across the state calls for this type of sampling. I see no reason this will create bias. Can someone please enlighten me?

EDIT: I also have landcover data, including satellite data the tells if a given area is urban, grassland, forest, etc. Wouldn't this make sampling as described obsolete?

Background on the model: I am modeling the abundance of a bird species as a function of land and observer effort covariates. Examples of covariates are: percentage of land type in a 2.5 kilometer by 2.5 kilometer region around the species observation (urban, forest, etc.), time species was observed, and time/distance traveled by observer that obtained the observation. Most raster cells don't have the bird (absence was recorded), so I'm using a zero-inflated Poisson model.

$\endgroup$
1
  • 1
    $\begingroup$ The answer depends on the circumstance. This is clear when you contemplate the myriad different possibilities: for instance, if the data are intended to characterize urban areas, the distribution must be uneven; but if the data are intended to characterize some property that extends across the entire square, it might or might not be appropriate to obtain an even representation. Overlaying a grid, btw, is a poor way to reduce sampling bias, because it throws away data. The right way is to use a statistical procedure appropriate for how the data were collected and for the objective. $\endgroup$
    – whuber
    Commented Feb 6, 2020 at 19:54

3 Answers 3

2
$\begingroup$

The basic problem is that individuals or households or whatever the observational units happen to be are not randomly distributed across space.

Consider for example individuals. To some extent people choose themselves where to live. However they do so according to their preferences (if your are an economist) hence certain people live in certain places and the people of one area are therefore not representative for the population of interest (as @Whuber points out this offcourse depend on what population you are interested in).

Consider a simple regression model

$$y_{i} = x_{i}^\top \beta + z_{c(i)} \lambda + \epsilon_{i},$$

where $z_{c(i)}$ is the characteristic $z_c$ of the area $c(i)$ that individual $i$ have chosen to live. Hence $z_{c(i)} := \sum_c z_c 1[i \in c]$ such that

$$y_{i} = x_{i}^\top \beta + \sum_c z_c 1[i \in c] \lambda + \epsilon_{i},$$

if the unobserved characteristics of the individual $\epsilon_i$ are relevant for where the individual choose to live then the indicator $1[i \in c]$ is assumedly correlated with $\epsilon_i$ hence $z_{c(i)} = \sum_c z_c 1[i \in c]$ is correlated with $\epsilon_i$ and the OLS estimation becomes inconsistent.

You could think of $z_c$ as the population density of the city and $y_i$ as individual worker wage. High density trigger mechanisms of agglomeration resulting in higher wages (imaginge some theory claiming that a causal effect of high density is higher productivity resulting in higher wages, perhaps something like big cities have airports and other high fix costs facilities that firms can share making them more productive, due to the high fixed cost such shared facilities do not appear in low density areas). This effect of density is assumed homogeneous across US-cities hence $\lambda$ measuring this efefct is not city specific.

However individuals with high unobserved skill (which is people with high $\epsilon_i$ - after all high $\epsilon_i$ means higher wage $y_i$ so the skills we do not observe and assume affect $y_i$ is in $\epsilon_i$ hopefully intuitive) are starting to return to the center of US metropolitan areas (high density $z_c$ areas). They choose to live in the high density centers of metropolitan areas perhaps because they are attracted by the aminities. Nevertheless this means that the observed $z_{c(i)} = \sum_c z_c 1[i \in c]$ becomes positive correlated with $\epsilon_i$.

The problem is then that when you measure the effect of density on wage you erroneously include a compsitional effect (big city labor force have a larger share of high skill labour, larger share of $\epsilon_i$) so part of the effect you attribute to density really appear becuase clever people to a higher extent prefer to live in the city rather than on the countryside (getting up early and milking the cow). The indicator is only there to make the choice explicit. What you want to do is to assign individuals randomly to cities (but short of dictatorship this is hard).

You should not read to much into this example. It is just a stylished example to illustrate how a non-random distribution of individuals across space can result in bias.

$\endgroup$
5
  • $\begingroup$ Could you specify what $\lambda$ is? And it seems that $z_c$ is ultimately some sort of local covariate, so why does each $z_c$ not have its own coefficient? And how is it possible to have an intercept for each city ($u_c$) if it is not a dummy variable? I guess I'm not sure what $u_c$ is either. I'm intrigued by your answer, but working through the notation. $\endgroup$
    – user271536
    Commented Feb 6, 2020 at 21:36
  • $\begingroup$ I made some edits and remove $u_c$ which was probably more confusing than helpful $\endgroup$ Commented Feb 6, 2020 at 22:37
  • $\begingroup$ This did clarify some things, and I think I get the gist. But it seems to me that one could deal with these issues by using an offset of population density. Additionally, if my goal is prediction (and it is), it doesn't seem like density falsely being construed as a causal factor in wage would be an issue. It's certainly a useful predictor. Right? $\endgroup$
    – user271536
    Commented Feb 7, 2020 at 0:24
  • $\begingroup$ What is an "offset of population density"? $\endgroup$ Commented Feb 7, 2020 at 0:56
  • $\begingroup$ More on offsets for GLMs: en.wikipedia.org/wiki/… $\endgroup$
    – user271536
    Commented Feb 7, 2020 at 1:16
1
$\begingroup$

Unevenly distributed data collection can (but will not necessarily) introduce a bias. This is a type of sample selection bias and is caused when the probability of a sample being generated is related to the quantity being observed.

This is easy to see with a simple example, suppose that you are collecting information on the rainfall across the region. Cities experience more rainfall than rural areas, so if you sample from city regions more frequently than other areas, then you will overestimate the amount of rainfall overall (at least if you just take a simple average across your samples).

Either way, I wouldn't ever recommend throwing out data unless you have a really good reason too! In most cases, you are able to correct for the bias introduced if you understand how your data was collected. In my rainfall example, one way this could be achieved by dividing your region into a grid as you suggest, but then estimating the total rainfall in each grid segment and summing the results.

In response to your edit:

I think you are getting overly caught up on the data you have available to you. Whether your model will produce biased estimates due to the uneven data collection depends upon the model / estimator being used not (just) the data available to you. you will need to provide more detail about your actual model for us to have any hope of telling if this is an issue in your case.

I should also correct a point i made above, as it has perhaps led to some of your confusion. The fix for the rainfall example i gave will only work if the probability for including each observation in your samples in a given grid segment becomes unrelated to the value of observation. You can easily see that this wouldn't hold if for example the grid segments still included both rural and urban areas (and may not even hold on purely rural or urban areas, this is something that requires subject matter expertise to solve).

$\endgroup$
2
  • $\begingroup$ Yes, I guess I left out the part that I have landcover data, including satellite data that for urban areas. Do you think that this resolves the issue? $\endgroup$
    – user271536
    Commented Feb 6, 2020 at 20:21
  • $\begingroup$ I added more details on the nature of the model. I'd be happy to provide more detail if you need it. $\endgroup$
    – user271536
    Commented Feb 7, 2020 at 0:12
0
$\begingroup$

This is the same case i am experiencing right now in my data, i collected a data in three different parts in my country and when i checked the specific bias, The results of chi-square test for zero constrained (χ2 = 86.114, df = 24, P < 0.000) and equal constrained (χ2 = 86.114, df = 23, P < 0.000) models were significant, meaning that the test of equal specific bias showed unevenly distributed bias.) After I searched i come to the conclusion that, when the simple size is greater 40, even when the data are not normally distributed that violation should not cause major problem. (Pallant J. SPSS survival manual, a step by step guide to data analysis using SPSS for windows. 3 ed. Sydney: McGraw Hill; 2007. pp. 179–200.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.