# Confusion about 1- vs 2-tailed tests for feature selection by hypothesis testing

Suppose $$x_i\ (i=1,2,...,N)$$ be attribute values for $$N$$ samples from class $$W_1$$ with mean $$\mu_1$$ and $$y_i\ (i=1,2,...,N)$$ be attribute values for $$N$$ samples from class $$W_2$$ with mean $$\mu_2$$.

For feature selection using hypothesis testing, why we should define $$H_0 =\mu_1 -\mu_2=0$$

• Comments are not for extended discussion; this conversation has been moved to chat. – kjetil b halvorsen Jan 22 at 17:01
• I agree with the book that (A) is correct. I'm pretty sure the TA is just wrong here. That happens. You'll probably have to talk to them to find out what their thinking was. – gung - Reinstate Monica Jan 22 at 17:12
• @gung-ReinstateMonica I think if the mean is not same, option (B) is the choice. isnt it? – Johnatan Morian Jan 23 at 23:17
• @JohnatanMorian, yes, that's right. – gung - Reinstate Monica Jan 24 at 1:34
• @gung-ReinstateMonica please see the last two slide on Page 1 (one) from this www2.newpaltz.edu/~liush/ST/ch10.pdf – Johnatan Morian Jan 24 at 10:53

This is one thought that $$(B)$$ is true:

You have two classes $$Z=0, 1$$ and usually we classify such that $$P(Z=1| X) >0.5$$. You need to choose a feature that increase the $$P(Z=1|feature)$$.

I think if you assume normality and use LDA (I mean assume variance of the features different classes are equal), then you should be able to relate the problem to the mean of the features. I also think 2 is the only correct answer if you look at the problem this way.

Suppose $$P(Z=1| feature) =$$ a $$\mu_{feature} + b$$ with $$a>0$$. Then $$P(Z=1|x) > P(Z=1|Y)$$ if $$\mu_x > \mu_y$$.

• I'm not sure I follow your solution here. Let's just talk about a single dimension for ease of exposition. Following notes from an introduction to statistical learning with R, LDA will assign a label of class 1 when $2x(\mu_1 - \mu_2) > (\mu^2_1 - \mu^2_2)$. This requirement does not make any assumptions re: the difference in the sign of means as per OP's question. Can you provide additional details? Additionally, if your expression $P(Z=1\vert feature)$ is indeed a probability, it does not appear to be bounded. – Demetri Pananos Jan 27 at 2:04

Option B is false.

I'm assuming the purpose of feature selection is to identify predictors which help distinguish between classes. Ostensibly, if two classes have an observed difference in class conditional expectation which is larger than what could reasonably happen under chance + test assumptions, then that predictor should be used to construct a model.

Assume that the sign of $$\mu_1 - \mu_2$$ did matter, and assume further that we would select feature $$x$$ because we reject the null of the test. The labels of the classes are arbitrary, so permute them switching $$W_1$$ to $$W_2$$ and vice versa. It is important to stress that nothing changes about the data itself, only the class labels.

Under this permutation, the sign of $$\mu_1 - \mu_2$$ would flip, and we would fail to reject the null and hence not select $$x$$ as a feature. That isn't a desirable property of the procedure. What features we select should not depend on arbitrary labels.

Its very clear here that your TA has made an error, and if the TA is insistent they are correct then they would have to deal with the dilemma I've outlined here: either the procedure depends on arbitrary labels, or the sign of the difference is irrelevant (unless we've made it relevant, in which case the question here is moot and the exam question is to blame).

You've already provided a reputable source, so I don't think additional sources are needed. You're right, the answer should be A.

• so would you please tell me what is wrong with my answer? another view to this problem – Johnatan Morian Jan 27 at 0:42
• @JohnatanMorian Immediately I see that you make some assumptions about a) classification rules, b) normality of the data, and c) the method of classification. You may be right, I haven't taken the time to check, but if you are right then you are right under a very narrow set of circumstances, none of which were mentioned in the question. It isn't a knock against you, I just don't see how this generalizes immediately. Can you expand your solution to be more general? – Demetri Pananos Jan 27 at 0:46
• so I'm not expert. would you please check my assumption? because you are expert in statistics. under my assumption is it right conclusion or not? if no I remove my answer. would you please check my inference using the mentioned assumption? – Johnatan Morian Jan 27 at 0:50
• @JohnatanMorian. Sure. Will check once I get home – Demetri Pananos Jan 27 at 1:03
• really appreciated. – Johnatan Morian Jan 27 at 1:34