Questions tagged [selection-bias]

Bias introduced by non-random selection of observations, such that the sample is not representative of the underlying population.

Filter by
Sorted by
Tagged with
1 vote
0 answers
19 views

Nested Cross-Validation with Small dataset

I am currently working with a small dataset (only 175 samples, 45 features) and have been reading on the proper way to cross-validate my model. I had started with a basic cross-validation using a grid ...
user avatar
2 votes
0 answers
25 views

Sampling weights in Cox proportional hazards models

I'd like to use sampling weights in a Cox proportional hazards regression model to address selection due to different response probabilities. I'm calculating the weights as Inverse Probability of ...
user avatar
  • 31
0 votes
0 answers
7 views

Estimation of the equivalence between 2 populations by using a Fisher's exact test

I am currently working on an article about a genetic disease caused by 3 groups of genetic mutation. A group of genetic mutation is particularly much more serious than the 2 others. In this article ...
user avatar
2 votes
0 answers
28 views

When can we get unbiased estimate given biased data?

There was a recent "hot take" tweet by Andrej Karpathy (without any comment or clarification from the author): real-world data distribution is ~N(0,1) good dataset is ~U(-2,2) It provoked ...
user avatar
  • 115k
1 vote
1 answer
52 views

Selection Models of Publication Bias for Multilevel Meta-analyses?

Are there any suitable selection models of publication bias for multilevel meta-analyses? I am currently conducting a 3-level meta-analysis and trying to incorporate selection models to assess ...
user avatar
  • 53
1 vote
1 answer
17 views

Sample Bias in Study

I have following Study statement: A council wishes to study the digital awareness of its resident senior population (over 65 years), so it questioned in person 50 residents randomly chosen from a ...
user avatar
  • 23
0 votes
1 answer
32 views

How to show how biased a statistic is in a non-random sample, knowing the parameter in the general population?

I have a convenience sample, and want to show readers how biased it is relative to the population it's taken from. It's absolutely certain that the sample is biased, and I want to give readers as many ...
user avatar
  • 127
1 vote
1 answer
48 views

Is it true that a larger, representative dataset is always better to use than a smaller, representative dataset?

By "representative" I mean that the data in the dataset faithfully reflects the "underlying signal" a model is trying to tap in to. Is it always true that, as long as increasing ...
user avatar
  • 121
0 votes
0 answers
13 views

Selection bias / Stratification - How to create a sample that represents the full population

I have built and deployed multiple models, initially starting with a heuristics model, eventually moving to various iterations of machine learning models where the target is binary. For each model, ...
user avatar
  • 121
0 votes
0 answers
32 views

Heckman model: satisfying exclusion criterion with control-variables

I have a situation where I need to use a Heckman selection model to correct for selection bias. Till now, I have specified my instrument as relevant for the selection equation, and that it has no ...
user avatar
  • 1
0 votes
1 answer
26 views

Can I ignore these individuals without introducing bias?

I have a population that falls under 10 classes. Each individual may or may not come with a location - 83% overall have locations and a breakdown by class is: Class # individuals # with location # ...
user avatar
1 vote
0 answers
35 views

What are the statistical fallacies of illusion of control?

Illusion of control* appears in gambling and events involving randomness. For example, choosing a lottery ticket which has an additional information that participant has a control of choosing, such as ...
user avatar
  • 1,997
0 votes
0 answers
35 views

Sample selection bias as internal bias under heterogeneous effects

I have been reading a lot about the extent to which sample selection biases that come from truncated samples (e.g., Heckman selection situations) are not only externally biased, which is more commonly ...
user avatar
  • 971
1 vote
1 answer
25 views

Estimating population means after a selection procedure

The problem is somewhat related to sequential procedures (e.g. Paulson; Gupta and Miescke) I have a hundred engines generating normal random numbers with true (unknown) means $E_1,E_2,E_3...E_{100}$, ...
user avatar
  • 13
0 votes
0 answers
12 views

Causal inference when selection bias affects response but not treatment

I have a problem where I need to estimate how variable X (risk indicator, a 0/1 "treatment" variable) affects variable Y (customer's risky behaviour). I have full knowledge of X for all ...
user avatar
1 vote
0 answers
16 views

Looking for a type of bias between "Selection bias" and "Framing"

I am looking for a word to describe a certain type of bias, something between Framing and Selection Bias. I am analyizing a dataset with measurements and their location in Berlin. The dataset includes ...
user avatar
  • 11
2 votes
0 answers
51 views

effect of vaccination against covid on hospitalization - help building a causal DAG [closed]

During the covid crisis, we have seen explanations of why vaccinated people end up less hospitalized than unvaccinated people under the paradox that there were more vaccinated people hospitalized than ...
user avatar
  • 325
0 votes
0 answers
27 views

Pre-study sample-pairing validity (two same-scaled IVs, binary DV)

Summary: is pairing samples ahead of time okay for testing? What test do you use given the resulting assumptions? Overview Goal: The goal is to determine which IV has a stronger influence on (...
user avatar
2 votes
0 answers
43 views

How to accommodate endogeneity after matching?

I am working on a field experiment where assignment to treatment vs. comparison was random, but participation uptake was not. The design is pre-post, and attrition is certainly not MCAR. This is a ...
user avatar
0 votes
1 answer
104 views

What is the difference between selection bias and composition effect?

Groups we seek to compare (e.g., a treatment group and control group) may differ in ways that constrain our ability to do so. Often, potential outcomes may differ systematically across groups, such as ...
user avatar
10 votes
3 answers
2k views

What is it called when an experimenter discards results that are too unexpected?

There is a type of scientific error where an experimenter gets a result significantly different from prior researchers, assumes they made a mistake, and redoes the experiment until they get a more ...
0 votes
0 answers
36 views

Heckman sample selection model

Instead of using a Heckman correction two-step solution, I used a one-step solution by means of multiple equations via SAS PROC QLIM. Is it true that when using a Heckman correction model all ...
user avatar
3 votes
2 answers
64 views

How to avoid selection bias while updating lead scoring (predictive) model with new data

We developed a standard lead scoring model using logistic regression on couple of months worth data. The model has been working and we have been pushing only top 1/3 leads to sales team basis that. ...
user avatar
0 votes
1 answer
28 views

Treatment Selection based on Treatment Effects

In many observational studies, researchers assume that the treatment decision is based on observable covariates (that is, the ignorability assumption). In this case, there are diverse methods that can ...
user avatar
  • 531
1 vote
0 answers
30 views

Why does the best fit model (lower AIC) yield higher p values than models with higher AIC? [closed]

Background: I am running a model selection in R that includes 1, 2, and 3-covariate models. Each model aims to determine the effect of environmental covariates in the occupancy of different species in ...
user avatar
0 votes
0 answers
18 views

How to handle retraining after model introduces bias

Here is my problem. I am retraining a machine learning model to detect fraudulent purchases. The training data is based on purchases and the target is whether or not they the purchase was considered ...
user avatar
1 vote
0 answers
94 views

Causal Inference: Selection Bias and Endogeneity [closed]

I've encountered lots of causal inference terms and jargons (under the Neyman-Rubin potential outcome framework), and I had questions regarding their relationships: I know that exogeneity E(e|X) = 0 ...
user avatar
  • 173
1 vote
0 answers
36 views

When we up-sample the training set, don't we introduce selection bias?

When doing supervised machine learning in the health or medical domains, we often have a target class that is relatively rare (e.g., prevalence 1-10% of cases). There are a few techniques we can do to ...
user avatar
0 votes
2 answers
58 views

Regression with endogenous dummy variables

Suppose I have 1,000 observations on variable Y, and I regress Y on just a dummy variable (d), in two different regressions. In the first regression, I set d equal to 0 for the first 500 observations, ...
user avatar
2 votes
0 answers
100 views

Which DAG would explain the lack of correlation between height and performance in NBA players?

A classic example of "selection bias" involves looking at the performance of professional basketball players. The example goes, among NBA players there is no correlation between height and ...
user avatar
  • 1,288
2 votes
0 answers
36 views

Using a sample of paid survey respondents to bias correct lower response rate among larger non-paid sample

I'm running a tracker survey on a website that has a low response rate of about 2%. The survey is not incentivized but the website traffic is large enough in volume I can always meet my sample target ...
user avatar
  • 121
1 vote
0 answers
97 views

Logistic regression for case-control studies

If I have designed a study where participants from 3 disease groups of fixed size were being sampled and suppose the three groups A, B and C are of sizes n_A=50, n_B=50 and n_C=100. Group A is a ...
user avatar
  • 311
2 votes
0 answers
40 views

Intended selection bias

Sampling or selection bias is often presented as something that has to be overcome, avoided, or at least appropriately considered because it's a problem otherwise. I wonder how often situations arise (...
user avatar
2 votes
1 answer
28 views

Implicit Selection Effects in Mixed Models

I'm struggling with a question that I haven't seen talked about anywhere. Potentially it just illustrates some ignorance I have about how effects are estimated in mixed models. Let's set up the ...
user avatar
2 votes
1 answer
60 views

Is self-selection for a treatment a problem after all?

I have devised the following R code. In this simulated dataset, we know that a certain treatment (e.g. a degree, an education) increases income by $ 1,000. Income is also caused by age. In the first ...
user avatar
1 vote
1 answer
344 views

Is it useful to implement clustered SE in the probit-type models?

For my research, I am implementing a two-stage Heckman procedure. I am working with panel data, so I was wondering if it is common and actually needed to use clustered standard errors for the first ...
user avatar
  • 27
0 votes
0 answers
63 views

Lagged dependent variables: Is it safe to ignore bias over a long time horizon?

Suppose I have a Poisson model which investigates the effect of a county level policy on robbery counts. Here is the basic specification: $$ \text{log}(y_{it}) = \theta y_{i,t-1} + \sum_i\text{County}...
user avatar
4 votes
2 answers
70 views

How would a statistician describe the problem with the figure in this publication? the solution?

I pointed out a problem with averaging values over time here https://www.researchgate.net/publication/344137839_SARS-CoV-2_binds_platelet_ACE2_to_enhance_thrombosis_in_COVID-19/comments in the ...
user avatar
2 votes
1 answer
173 views

How does the support vector machine constraint imply that sample selection bias will not systematically affect the output of the optimisation?

I am currently studying the paper Learning and Evaluating Classifiers under Sample Selection Bias by Bianca Zadrozny. In section 3.4. Support vector machines, the author says the following: 3.4. ...
user avatar
  • 1,126
0 votes
1 answer
30 views

What is meant by "the number of examples is reduced", and why is this the case?

I am currently studying the paper Learning and Evaluating Classifiers under Sample Selection Bias by Bianca Zadrozny. In section 3.2. Logistic Regression, the author says the following: 3.2. Logistic ...
user avatar
  • 1,126
4 votes
1 answer
153 views

$\frac{P(x_1 \mid y, s = 1) \dots P(x_n \mid y, s = 1) P(y \mid s = 1)}{P(x \mid s = 1)}$ indicates that naive Bayes learners are global learners?

I am currently studying the paper Learning and Evaluating Classifiers under Sample Selection Bias by Bianca Zadrozny. In section 3. Learning under sample selection bias, the author says the following: ...
user avatar
  • 1,126
1 vote
1 answer
19 views

How to estimate the impact of policy on labor supply if the policy causes death, too?

Suppose people can make only one of three choices A, B, C. A = Death B = Survive, Work C = Survive, Not work I want to study the impact of an exogenous policy on ...
user avatar
1 vote
0 answers
85 views

Is this referring to the true underlying distribution, or the distribution of our sample?

I am currently studying the paper Learning and Evaluating Classifiers under Sample Selection Bias by Bianca Zadrozny. In the introduction, the author says the following: One of the most common ...
user avatar
  • 1,126
0 votes
1 answer
102 views

Endogenous Sample Selection and Heckit Correction in relation to Research Question

I am researching the funding amounts of start-ups. I am calculating two models: ...
user avatar
  • 1
0 votes
0 answers
15 views

Truncation based on two variables

I have some data, which has been constructed from multiple API pulls. Observations have an expiry date, $x_i$, which means that observations which expire before the API pull is done do not appear in ...
user avatar
11 votes
3 answers
559 views

Abraham Wald survivorship bias intuition

During World War II, the statistician Abraham Wald took survivorship bias into his calculations when considering how to minimize bomber losses to enemy fire. Wald noted that the study only considered ...
user avatar
  • 731
0 votes
1 answer
259 views

Heckman Selection (Selection vs. Outcome equation)

A common example of the Heckman selection model involves wages, which are only observed if an individual chooses to participate in the labor force. The first stage probit model dependent variable is a ...
user avatar
  • 11
1 vote
2 answers
74 views

How to solve selection bias in a survey?

Selection bias often happens in survey studies. For example, sending out a survey to 10,000 customers to ask whether they like one specific product or not. Only 10% of people respond and 80% of them ...
user avatar
  • 119
3 votes
1 answer
700 views

Heckman with second step probit in R [closed]

The functions selection and heckit (package sampleSelection) support a binary dependent ...
user avatar
  • 31
1 vote
0 answers
154 views

Selection bias in categorical independent variable in regression analysis

I have an issue with selection bias in my independent variable and I am uncertain of the best approach to correct for it. I'm looking at soccer data and trying to make some forecasts on players from ...
user avatar