2
$\begingroup$

The real question here is if there is a way to know X is causing Y rather than simply being associated with it. In my example Y seems to be telling more of X(if not causing it) than X says about Y.

This is not about causality per se, I know that correlational studies can not show that. But we will never be able to do random assignment, it would not even be legal. So correlation is what we have. My problem is that I am having growing doubts about the use of regression to show that X causes a change in Y. For example in our survey of satisfaction we measure a wide range of predictors of satisfaction, like satisfaction with pay, and then use it to predict overall satisfaction. Satisfaction with pay (a dummy predictor variable) has the highest odds ratio in the survey (the DV is a two level variable satisfied/not - I am running logistic regression). If you are satisfied with pay you are 19 times more likely to be overall satisfied than if you are not satisfied with pay. The odds ratio suggests that this is a key factor in overall satisfaction therefore at least compared to 30 other predictors.

The problem is that there is little indication this really matters. 90 plus percent of our staff are satisfied (that the number is so high might be part of the problem because there are only 47 usable cases that are dissatisfied out of 448 total cases - also we have 31 predictors). And dissatisfaction with pay is extremely high. So pay looks like a key driver of overall satisfaction based on the odds ratios, but satisfaction with pay is very low and overall satisfaction is quite high. It seems that while overall satisfaction reflects real differences between those who are satisfied with pay and not, those differences don't change whether customers are satisfied or not. Which confuses me given my understanding of regression.

It is interesting that two variables with much lower odds ratios have higher levels of statistical significance. Pay is significant at the ,1 level they are at the .05 level.

$\endgroup$
4
  • $\begingroup$ It's possible that your target causes one of your predictors. I don't think there's a magic bullet to tell if this is true. A (maybe) relevant example: I used to work in bars and restaurants. The pay isn't great. Few would answer yes to the direct question "Is your pay satisfactory", and everyone knows this. BUT, employees generally only actively grouse about the pay when they're unsatisfied with the job more generally. Target - > predictor. $\endgroup$ Sep 17, 2021 at 4:16
  • $\begingroup$ If dissatisfaction with pay is extremely high, and you live in a free country (or at least a relatively free country), then your dissatisfied employees will vote with their feet: out the door. I personally think dissatisfaction with supervisors is more common than with pay. People tend to leave bad bosses at least as much if not more than bad pay. $\endgroup$ Sep 17, 2021 at 16:02
  • $\begingroup$ Hi @user54285. I'm studying a course which deals with similar issues to your question. I would like to note the difference between an observational study and an experimental study. The former can only provide you with suggestions of associations, while the latter can give you information about causation. You can see texts on Experimental design that make this clear. From what I understand in your example, your predictors (satisfaction with pay) are observational, and not in your control, so you can only infer association with your response. There may or may not be a confounding variable. $\endgroup$
    – user523384
    Sep 21, 2021 at 10:48
  • $\begingroup$ From what I know, I don't believe you can use a special technique to extract causal conclusions from your data. I'm only an undergrad, so hopefully someone more experienced can confirm this. But your doubts about regression seem reasonable to me. A statistically significant relationship between X and Y does not necessarily mean X causes Y, it just suggests they move together. However, it also means we are unsure if setting X to a particular value will cause the same change in Y, as opposed to letting nature set X and Y and just "observing" it. $\endgroup$
    – user523384
    Sep 21, 2021 at 10:53

1 Answer 1

1
$\begingroup$

I'm a little confused when you say both that "The real question here is if there is a way to know X is causing Y rather than simply being associated with it." as well as "This is not about causality per se, I know that correlational studies can not show that." This is definitely about causality.

Theorem 1.2.8 (Observational Equivalence) in Pearl's book Causality: Models, Reasoning, and Inference, p. 19, runs like this:

Two DAGs are observationally equivalent if and only if they have the same skeletons and the same sets of $v$-structures, that is, two converging arrows whose tails are not connected by an arrow.

This implies, among other things, that the DAG $X\to Y$ is observationally equivalent to $Y\to X:$ these two DAGs have the same skeleton and the same $v$-structures. Therefore, there is nothing at all you can do to distinguish between the two.

In general, the only thing you can really rely on in a situation like this is human intuition or expert opinion.

$\endgroup$
4
  • $\begingroup$ What I meant is that causality in the classic sense can't be proven because I did not do random assignment and can not. I wondered if there were any ways you could be more sure (I know some theoretical ways like regression discontinuity designs, but I have not done them from graduate work - they are rarely used). I don't know what DAG means or skeleton or v structure. I assume you are saying that whether X is driven by Y or Y by X (or either) really can only be known based on theory or inference. But I think that is where I am having to use intuition. I just hoped there was another method. $\endgroup$
    – user54285
    Sep 17, 2021 at 1:17
  • $\begingroup$ I am a practitioner rather than a statistician. So I look at regression fairly simply. I assume if the mean difference on Y between two levels of a dummy X are large this suggest that something about X pushes Y in a specific direction. I always have. But looking at this data makes me honestly wonder. It is very unlikely logically for satisfaction with pay to drive overall satisfaction significantly because then satisfaction would be much lower. But it has a very high odds ratio. I was hoping for some trick to get around this. Thanks. what I need maybe is multivariate descriptives. :) $\endgroup$
    – user54285
    Sep 17, 2021 at 1:35
  • $\begingroup$ Hi @user54285, I made a comment above. To share my understanding here, I think a factor X can still have a high odds ratio with Y without actually causing it. There might just be a confounding variable that raises X and Y together. There may be various other factors, such as number of close relationships, which may raise overall satisfaction and satisfaction with pay together (the person may just be more content with less). Hopefully these comments may be helpful, again just a disclaimer I'm only an undergrad! $\endgroup$
    – user523384
    Sep 21, 2021 at 11:00
  • $\begingroup$ Glad to see someone other than me given disclaimers. :) There could be some variable driving both not in the model, always a problem. I am more interested in finding something to test causality in this case, but I think now this can only be done with theory which sadly does not really exist in this case. Experimental design, that is not an observational study, would be the ideal but as with most government issues random assignment is not even legal with my question. So correlation is all you have. $\endgroup$
    – user54285
    Sep 21, 2021 at 21:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.