Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 1 down vote favorite
1

I have been searching on the net for the term dependent and independent samples, but I couldn't find proper definition, nor to conclude what exactly it is. It would be nice if anyone of you could explain it and bestow us with the knowledge.

share|improve this question
typically people refer to non independent samples as being correlated, en.wikipedia.org/wiki/Correlation_and_dependence – anonymous_4322 Apr 11 '12 at 1:48
Some more information would be helpful. But, in the case of a t-test, dependent samples usually refer to 'paired' samples (like before/after measurements on a set of individuals). Independent samples are just the way they sound. Based on @anonymous_4322's comment he/she interpreted your question as a matter of a sample of individuals that are independent of each other vs. a sample of correlated data. Perhaps you can clarify the context a bit more? – Macro Apr 11 '12 at 2:01
It already sounds complicated. :) Yes, I was thinking more on the types of data, so if you could evaluate your answer more on that matter, it would be great. – Serenity Stack Holder Apr 11 '12 at 5:16
1  
Also, "dependence refers to any statistical relationship between two random variables or two sets of data" is such a notoriously explained that I am confused at the start. Based on what can we conclude that two samples are dependent on each other? – Serenity Stack Holder Apr 11 '12 at 5:24
1  
No, dependence has nothing to do with whether the data are continuous or not. – Peter Flom Apr 12 '12 at 22:16
show 1 more comment

3 Answers

up vote 1 down vote

@whuber is right that we need a little bit more context to decipher what you mean by "samples." If you mean "samples" in the sense of "the result of doing sampling," and thus you're using the term as a synonym of "realizations", then the following applies:

Samples are dependent conditional on some (or possibly no) prior knowledge if and only if knowing something about one sample could tell you something new about the other sample.

The most common case is where samples $x_1, x_2$ are assumed to be "independently and identically distributed" according to a distribution $D$. In that case, given that you know $D$ is say a normal distribution with mean zero and variance one, knowing that the value of $x_1$ is $1.2$ your belief about $x_2$ is still that it follows $D$. Without knowing $D$, however, but knowing that the samples were iid from some normal distribution, leaves the samples clearly dependent: knowing something about one tells you something about $D$, which tells you something about the other.

Without assuming dependence or independence between samples, it's impossible to know whether they are dependent or independent, but one can often make good guesses by trying to find patterns. Correlation, the example above, is just one such pattern.

share|improve this answer
(1) Because this reply is clearly from a statistical standpoint, it would help to use conventional statistical terminology: your "sample" is usually called an outcome. (2) That middle paragraph sounds wrong: an iid set of outcomes from a common but unknown distribution is still iid, regardless. You appear to confuse independence with predictability. – whuber♦ Apr 11 '12 at 19:15
@whuber: I think you're right about sample vs. outcome. But, regarding (2): If you know that outcomes $x_1, x_2 \sim D$, but you don't know $D$, then $x_1$ and $x_2$ are dependent. You can draw the graphical model and examine that they are d-separated when $D$ is known, and d-connected when it's not. – Neil G Apr 11 '12 at 19:25
1  
(1) "A sample is a set of items or individuals selected from a larger aggregate or population about which we wish quantitative information." Snedecor & Cochran, Statistical Methods, Eighth Edition (p. 5). (2) "Independence" in the sense of independence of random variables is not defined in the sense you use it here--for that, I think I can safely refer you to any authoritative statistical source, even Wikipedia. – whuber♦ Apr 11 '12 at 22:01
1  
Now you get my point (which I also made in comments to the reply by @cbeleites): I believe a good answer to this question is impossible to formulate until either the question is revised and clarified or the answer begins by defining its terms clearly. I hope you might find it possible to improve your reply in that way. – whuber♦ Apr 12 '12 at 13:39
1  
@whuber: Agreed. – Neil G Apr 12 '12 at 16:28
show 4 more comments
up vote 1 down vote

Reading all the answers and comments, it's clear that we have a bit of a Catch-22 here. People can't answer the question without more context, but the question seems to be asking for that context.

So, I'm going to take a shot at this, trying to guess what Serenity Stack Holder means.

Two samples (or more than two) are dependent if they are somehow connected, not by having similar results necessarily, but by having one result in some way depend on the other result. For example, suppose I am interested in comparing the heights of men and women. If I randomly pick 50 women and 50 men from some population, the samples are independent, because what one person's height is has no bearing on another person's height. One thing gives you no information about the other. However, if I picked 50 heterosexual couples, the two samples would not be independent, because people tend to marry people of similar height.

I hope this helps!

share|improve this answer
up vote 0 down vote

terminology:
I'm chemist. I have many samples which together form one sample in the statistical sense.
see also: How to define what a "sample" is?

Maybe a list with easy cases is a start:

  • if your samples are correlated, they are not independent (but you cannot conclude the other way round).

  • (kind of obvious): if one sample influences the other, they are not independent

  • if you know a cause that influences both samples, they are not independent

  • if you are able refine a prediction of whatever you are interested for one sample once you know the result for another sample (i.e. better than guessing from the overall distribution), then samples are not independent.

  • it is always difficult to argue independence: Imagine you study hair color and to do that you grab people every 30 min from the street in front of your office. You may consider the persons independent: no way to predict another persons hair color better than guessing the average hair colour. But how do you know (prove!) that you did not just miss the right model to predict hair colour?

  • (In)dependence may be discussed with regard to the scope of the study: now your colleague at the other side of the globe joins your study and sends you hair colors of people in front of his office. Now you can predict upcoming hairs better than general guessing from the last hair color you collected: hair colour is not independent of geographical region.
    You may say that the population to be studied needs to be well defined in order to argue (in)dependence: is it "hair colours appearing in front of my office" or is it "hair colour of humans"?

share|improve this answer
(1) In statistics, a "sample" usually refers to a related set of data, usually obtained in a clearly specified way. You appear to use "sample" differently here. What exactly does it mean to you? – whuber♦ Apr 11 '12 at 18:14
@whuber: Sample (material). How do you refer (in a way that statisticians, chemists and normal people understand) to one object(?) in your sample? As I mainly work with biological or medical samples, maybe specimen is a good term (but I've never heard that wrt. analytical chemistry)? – cbeleites Apr 11 '12 at 18:26
1  
"Sample" has a clear and conventional meaning in the empirical sciences but it differs from the statistical one. When both senses are needed, I have found that referring to the physical "sample" in terms of its measurements or as an observation often leads to clear language. This brings us to another (hidden) issue: "independent" means distinct things, too. It makes no statistical sense to refer to physical things as "independent," but such things could have been (say) randomly selected, which is a kind of physical independence, implying statistical independence of its measurements. – whuber♦ Apr 11 '12 at 18:41
@whuber: I agree that this can solve the problem. Unfortunately, for the kind of experiments I do there's an additional level: I take many (usually spatially or temporally resolved) measurements of each sample. I commented on the sample terminology question. – cbeleites Apr 11 '12 at 18:47
2  
You can address this at the outset by defining your terminology. I'm sympathetic: your case is like mine (environmental sampling), where it's convenient to say that a "dataset" consists of "observations" of individual physical "samples," each of which is subjected to one or more analytical "tests" that typically yield multiple "measurements." Notice how the statistical terminology is largely dropped in favor of words familiar to the reader: the burden of good communication lies first with the writer, who should reach out to the audience and build on their conventions and understandings. – whuber♦ Apr 11 '12 at 19:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.