Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

I have two variables and 1000 cases. How can I statistically find representative cases from total of 1000, based on statistical properties of both variables and correlation between them. Perhaps something based on T-test and 95% (or 99%) interval but for both variables? I would like to know which statistical method can find cases that have both values (simultaneously) statistically the most significant. I know that this deals with sample distribution and estimating the proportions.

share|improve this question
2  
T-tests, significance, sample distributions, and estimates of proportions deal with conclusions you draw from the data. Finding "representative cases" is a matter of selecting cases from the dataset. The two activities are completely different. What, then, do you really want to do? – whuber♦ Mar 10 '11 at 15:30
Thank you for your answer. Indeed, I would like to find such cases that both data measurements (two variables) are in 99% (or 95 or 66) interval of their individual distributions. So it could be something like intersection (Boolean logic) of very significant data of two variables. Correlation between variables is 0.35275, one variable is we can say dependent on other but there are exceptions. – user3656 Mar 11 '11 at 10:16
You just answered your own question: compute the interval endpoints and make the selection. But your references to "significant" and the implication that this procedure is "representative" are disconcerting because they suggest you are looking for an automatic or mindless way to discard the most interesting--and perhaps the most important--cases in your dataset; namely, those that don't lie in both intervals. The wisdom of that approach depends on what use will be made of the selection, which you haven't revealed. – whuber♦ Mar 11 '11 at 15:30
Aim of my whole procedure is: I will use selection for some further procedures in measurements. So at the beginning i have 1000 cases and six variables, after factor analysis, deleting some cases with one variable value and outlier test i get 53 cases and two variables. So i would like to know 99 95 or 66% confidence interval for variable 1 and 2 and select 5 to 10 cases for further work. – user3656 Mar 11 '11 at 21:49

1 Answer

up vote 0 down vote

I like multivariate Gaussian Mixture Models because, with just a few dimensions to the data, they can show something about the interaction between the values.

Here is an example of a Gaussian Mixture Model in plot. http://www.mathworks.com/help/stats/gaussian-mixture-models.html#bra9fvn

When you say "most representative" to me that says "expection" or "mean". The figure shown in step 3 of the preceding link gives you the two modes/peaks of "higher expectation". If you were looking for central tendency then you should pick there.

I think that you are looking for domain-spanning sub-sampling. It sounds like you want data, not only at the edges, but with some measure of "uniformity" across the potential values. This could be for some experiment or characterization. If you know something about the actual distribution by fitting or knowledge of the system that produced the data then you could use methods from "design of experiments" to determine which samples are most informative. A useful sampling strategy might be to find "D-optimal" sampling locations within the domain.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.