How to find statistically significant results without doing an experiment? Suppose I want to determine the effect of an intervention on a health score. I do not have the means to design an experiment to randomly assign people to the control and treatment groups. The way I recruit people to apply for the intervention is by mailing advertisements to their homes, or going to hospitals and giving seminars about the intervention. After a month, I want to measure the mean health score for people who signed up for the intervention vs. the rest of the population see if there is a statistically significant difference. How would you apply the standard ANOVA/t tests if the samples were not random?
 A: You apply the standard test that you would have applied had you randomized. Statistical significance tells you if you have evidence that the groups differ in the population on the outcome. Then you have to interpret that result. 
There is a whole literature on causal inference. The Wikipedia entry on causal inference starts: "Causal inference is the process of drawing a conclusion about a causal connection based on the conditions of the occurrence of an effect." Currently, you have the condition of the occurrence of an effect - you want to know if that difference has occurred because of your intervention - i.e. is the effect causal.
Determining whether a relationship is causal is a much bigger question than can be answered on a forum like this - they have been being debated for more than 50 years, books have been written on the subject, and other books have been written that disagree with those books. There's a whole journal devoted to the subject: https://www.degruyter.com/view/j/jci
However, the way that I sometimes think of it is to say:


*

*I have found an effect. 

*Maybe it's causal, or maybe there's another
mechanism. Can I eliminate this other mechanism. If so, try.

*Is there another potential mechanism? If so, return to 2.

*The more alternative mechanisms I've eliminated (and I can never eliminate them all, because I haven't thought of some of them), the more confident I am that I have found a causal relationship.


That's something of an oversimplification - some of the references in the Wikipedia entry are good starting points. If you make some progress with these, you'll probably be able to come back with a more specific question.
A: An alternative answer is that a simple ANOVA/ANCOVA adjusting for pre-intervention score/t-test will not tell you whether the intervention does anything,  because people that use it may differ from those that do not for reasons other than the intervention. Very often such differences are bigger than the intervention effect,  if any.
Attempts to deal with that include things like analyses stratified by propensity score - a score/probability  for choosing the intervention estimated using all information that affects the decision (if you do not have some of that information that puts the analysis in question) - and other similar analysis approaches.
