Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [simpsons-paradox]

Simpson's Paradox is an example of the Reversal Paradox, where an association appears in several different groups of data but disappears, or even reverses in sign, when these groups are combined.

3
votes
0answers
39 views

Looking to identify a book by a top statistician with a chapter on Simpson's Paradox

It was more than 20 years ago. I had just gotten acquainted with Simpson's paradox. I was browsing in a bookstore and saw a book by an eminent statistician -- eminent in the sense that I had come ...
1
vote
1answer
45 views

Example distribution to match an example?

I am doing exercises from "Causal Inference in Statistics: A Primer", by Pearl et al (2016). In chapter 1.2 there is a training challenge that goes like: In an attempt to estimate the effectiveness ...
0
votes
1answer
36 views

When should we use the segregated as opposed to the aggregated data?

In the book "Causal Inference In Statistics" by Pearl et al., there is the following problem (study question 1.2.2.) A baseball batter Tim has a better batting average than his teammate Frank. ...
1
vote
0answers
21 views

Recovering hidden confounder in Simpson's paradox trends

I just watched a video of an interested talk from PyData LA: "Using Simpson’s Paradox to Discover Interesting Patterns in..." - Nazanin Alipourfard, Peter Fennell (https://www.youtube.com/watch?v=...
5
votes
2answers
139 views

When does Simpson's Paradox “end”?

Disclaimer: This is not a duplicate of How to resolve Simpson's paradox. As given in this blog, the following is the data of people on the titanic: This is the same data when divided on basis of ...
1
vote
1answer
139 views

Detecting Simpson Paradox using lme

I am playing with a toy data where the Simpson's paradox exists for two variables NO2 and temperature: A scatter plot clearly shows that the correlation between NO2 and temperature was reversed when ...
10
votes
2answers
967 views

Does Simpson's Paradox cover all instances of reversal from a hidden variable?

The following is a question about the many visualizations offered as 'proof by picture' of the existence of Simpson's paradox, and possibly a question about terminology. Simpson's Paradox is a fairly ...
2
votes
2answers
521 views

Intuition needed when using weighted average to explain Simpson's paradox

In Freedman's Statistics (chapter 2), the author uses Berkeley's admission statistics (that 44% men and 35% women were admitted to graduate programs in general) to illustrate Simpson's paradox: the ...
1
vote
0answers
16 views

Language data as as a ratio of the number of words?

I am analyzing language data (nouns, verbs, etc) of people with and without autism and all of them have different number of words. Option 1: Would it be best to count each result as a ratio of the ...
0
votes
1answer
864 views

Effect of combining predictor variables in a regression model

Let's say I first run a linear regression model Sales = f(TV Spend, Digital Spend). Now I add TV Spend and Digital Spend and run the second model. My second model is Sales = f(TV Spend+Digital Spend)...
3
votes
1answer
72 views

Any examples of generalisation of simpson's paradox to other metrics

Simpson's paradox is introduced on wikipedia using 'metrics' of success rates and regression coefficients, for the first (success rate of kidney stone treatments): How to resolve Simpson's paradox?...
14
votes
2answers
656 views

Can you please explain Simpson's paradox with equations, instead of contingency tables?

I probably don't have a clear understanding of the Simpson's paradox. Informally I know that the average of response Y1, grouped over all possible levels of factor A, can be higher than the average of ...
4
votes
2answers
290 views

Is Simpson's Paradox always an example of confounding?

Is Simpson's Paradox always an example of confounding? Or is it possible to have a Simpson's paradox effect without an extra variable lurking in the background?
3
votes
0answers
96 views

Is stagewised feature engineering/ selection an invalid approach? What to do when all the features are not ready at one time?

Suppose we want to build a regression or classification model. However, the features (independent variables used) are not all ready at one time. This is very realistic in business, because the data ...
3
votes
2answers
725 views

A way to show Simpson's paradox is happening in the data

I'm not entirely sure what I am talking about is Simpson's paradox, because an opposite relationship does not appear when you combine two data sets, but merely a different one. Still, I think it is ...
7
votes
1answer
1k views

Is this Simpson's Paradox on the Titanic data set?

With the well known "Survival of passengers on the Titanic" data set I get a strange behaviour by plotting the fare vs. the age (see below). Without a constraint on Pclass the correlation is positive. ...
5
votes
1answer
2k views

Simpson's Paradox & Random Effects

Simpson's effect is key to understanding potential pitfalls in medical research, particularly in the area of meta-analysis, where multiple studies with dissimilar methodologies are brought together to ...
2
votes
1answer
130 views
3
votes
1answer
56 views

What to look for to spot potential cases of Simpson's paradox?

I understand the basics of Simpson's paradox, but I'm not yet confident that I'd always be able to "avoid" it, or to spot cases where others have failed to do so. To be more precise, I'm not sure I ...
8
votes
2answers
598 views

Does adjusting for superfluous variables bias OLS estimates?

The usual textbook treatment of adjusting for superfluous variables in OLS states that the estimator is still unbiased, but may have larger variance (see, for example, Greene, Econometric Analysis, ...
2
votes
0answers
104 views

Simpson's Paradox with new sample

Suppose you constructing model whose training data is cumulative in nature; meaning each year you can add new observations with all prior observations being kept the same. (e.g. training set is non-...
21
votes
3answers
1k views

Understanding Simpson's paradox: Andrew Gelman's example with regressing income on sex and height

Andrew Gelman in one of his recent blog posts says: I do not think counterfactuals or potential outcomes are necessary for Simpson’s paradox. I say this because one can set up Simpson’s ...
3
votes
0answers
562 views

Why signs of coefficients change when doing multivariate vs. univariate logit regression? [duplicate]

Excuse my dumb question, but I did an univariate logistic regression where the sign of the coefficient of my variable was negative (and it was significant). Once I have input it into a multivariate ...
2
votes
1answer
290 views

A question about resolving Simpson's paradox by normalization

I recently asked a question about Simpson's paradox. Suppose we are concerned about making some choice about an arbitrary element of a population. Recall that Simpson's paradox arises when the answer ...
35
votes
5answers
12k views

How to resolve Simpson's paradox?

Simpson's paradox is a classic puzzle discussed in introductory statistics courses worldwide. However, my course was content to simply note that a problem existed and did not provide a solution. I ...
1
vote
0answers
132 views

Simpson's Paradox, Combining data across confounding variable when few values are missing

The statistical analysis of experimental data that I have to perform could be described as follows. Three drug treatments $D_1$, $D_2$ and $D_3$ were tested across three groups $G_1$, $G_2$ and $G_3$. ...
3
votes
1answer
590 views

Does Simpson's Paradox affect if the researcher can conclude that alcohol consumption is related to gender?

I am preparing for my statistics final and have this question in the review package to answer: I understand the hypothesis portion of the question. How should I approach the first question: Can the ...
27
votes
3answers
4k views

Which hospital should be chosen? One has a higher success rate, but the other has a higher overall success rate

I have a question about something that my statistics teacher said about the following problem. My question isn't even about the occurrence of Simpson's paradox in this situation. My question is simply ...
4
votes
1answer
1k views

Simpson's paradox or confounding?

Consider a scenario where a two-way contingency table is analyzed by a chi-squared test of independence and a significant result is found. Now, it turns out that this table is an aggregation of data ...
4
votes
4answers
177 views

Feature selection for classification, controlling for sub-population

I have a bunch of points that belong to one of population P1, P2, ... Pn AND to class A or B. Within each population I'll be doing classification between A and B, and I want to select features that ...
6
votes
2answers
412 views

Question about combining hazard ratios - Maybe Simpson's paradox?

I’m reviewing an article, and can’t give details but here is the situation, and it’s got me puzzled Patients were divided into 4 categories (call them A B C and D), which were exhaustive and ...