Assess if 2 groups are different given control variables Say I have 2 groups of people, selected randomly. For each person, I know some characteristics about them, in the form of continuous and categorical variables. For example, that could be the person's weight (continuous integers), whether they smoked or not (binary variable), number of 10 min+ walks per week (1, 2, 3, 4+) etc...
At this point, I am interested in deciding wether the 2 groups look identical, according to my control variables stated above.
I know I can compare the 2 groups for each variable. I could perform a t-test on the group's weight distributions. I could run a chi-squared test when the variables are categorical.
I would like an overall test that will, at the end, give me one p-value describing the difference between the 2 groups and decide if I can consider them significantly different or not.
Does such a test exist? If not, how would you tackle this problem?
 A: If the purpose of this exercise is to assess balance for the purposes of comparing groups in an observational study, you shouldn't be using hypothesis tests to do so. This is commonly referred to as the "balance test fallacy". In a balance test, you only care about assessing whether the groups in your sample differ from each other. See Imai, King, & Stuart (2008) and Ho, Imai, King, & Stuart (2007) for a discussion.
If you are truly interested in making an inference about the population from which your sample was drawn, what you probably want is a multivariate t-test (MANOVA), or the logistic regression approach could work as well. One problem with the logistic regression approach is that it tests for conditional associations between each variable and the grouping variable, when you actually want marginal associations. The MANOVA makes some assumptions about normality.
A: At first, when there is random selection, then it is not needed to perform such tests.
In cases, where there is a reason to think, that selection was not random, but possibly the populations do not differ in features, we might build a selection model. First choice would be logit with group as dependent variable. Testing for significance of whole model would give the right answer, as long you have chosen good variables.
It is actually necessary to have one test for all the comparisons, because we avoid multiple comparison problem this way.
To perform joint test for significance of all variables in logit model, first-choice test would be Likelihood-Ratio test. Some statistic software does it automatically, when you estimate the model. One can thing of this test as equivalent of the F test for OLS regression.
