# Fisher Distance for feature selection

I'm currently working for EEG signal classification from 3 electrodes. I want to have a simple feature selection algorithm that is independent with the classification process. From the feature extraction step, let's say I have this kind of matrix now (not the actual numbers/data) :

A CLASS : $$Ch_1 = \begin{vmatrix} 1 & 2 & 3 \\ 0.5 & 0.2 & 0 \\ 1 & 0.1 & 0.8 \\ 1.2 & 0.8 & 1 \end{vmatrix} Ch_2 = \begin{vmatrix} 1 & 1.5 & 1 \\ 0.3 & 0.1 & 2 \\ 1.3 & 0.1 & 3 \\ 1.5 & 1.8 & 2 \end{vmatrix} Ch_3 = \begin{vmatrix} 2 & 2 & 3 \\ 1.2 & 2 & 0.8 \\ 1.3 & 1.2 & 1.5 \\ 1.8 & 3 & 2 \end{vmatrix}$$ B CLASS : $$Ch_1 = \begin{vmatrix} 1 & 2 & 3 \\ 0.5 & 0.2 & 0 \\ 0.1 & 2 & 0 \\ 1.2 & 0.8 & 1 \end{vmatrix} Ch_2 = \begin{vmatrix} 1.2 & 1.5 & 1 \\ 0.3 & 0.1 & 2 \\ 0.8 & 1.1 & 0 \\ 1.5 & 1.8 & 2 \end{vmatrix} Ch_3 = \begin{vmatrix} 2 & 2 & 3 \\ 1.2 & 2 & 0.8 \\ 0.2 & 1 & 0.3 \\ 1.8 & 3 & 1 \end{vmatrix}$$

Where on the example above, the row of the channels are the numbers of trials/observations (4 trials per class) and the column are the features extracted from each sub-band (3 features).
What I want to do is selecting which feature will give me better separation of data between classes, while maintaining close relationship within its own class.
I am trying to do Fisher Distance approach : $$FisherDis = S_B/S_w$$ Where $S_B$ is between class matrix and $S_w$ is within class matrix. From what I read, I have to score each feature and then select some features with highest scores.

Now to my question:
1. What is "the number of samples" when I want to calculate $S_w$ and $S_B$ , is it four (as in four trials) or three (as in three features) ?
2. Should I group the channels into one matrix? Or is it better if I'm working in each channel separately?
3. Am I working on the right path? I have doubts in myself...

Thank you very much in advance. I'd appreciate every answer from everyone because I'm fairly new to statistics (I have so much to learn..) :)

1. 'N' or number of samples is usually the number of cases, this can be the number of subject (assuming you have one measurement of each feature per subject) or the number of measurements per feature. If you have multiple measurements per feature, per subject you will need to account for this.
2. Generally, EEG channels should be considered separately, as different activity patterns can be observed in different locations, i.e. occipital alpha activity may have different characteristics to frontal alpha for a given subject. For this reason I would recommend calculating each feature separately for each channel. IF it is justifiable for the application you can average each feature across all channels for each subject.
3. One solution could be to take the mean value of each feature across all four trials, calculated for each of the 3 channels (electrodes) giving 9 features for each subject.

You can use hypothesis testing such as a t-test or rank-sum test to examine how well each feature distinguishes between the two classes. However, if you are performing classification, feature selection should be performed within each fold of cross-validation.

• first thank you for the answer, so what I should do is find the covariance (for the within class scatter) between each features (with each features contain n subjects), is this right? thanks
– Neu
Nov 1, 2012 at 0:28
• It depends on the application but the standard thing is to find the within class scatter (covariance) and the between class scatter, with each subject represented by one sample in both Nov 1, 2012 at 8:37
• I don't really get the 'each subject represented by one sample' part... and when I want to find within class scatter, does the 'class' means the features (3 in my example) because I want to see which feature is 'close' with itself in each trials? Thanks
– Neu
Nov 1, 2012 at 12:33
• Class is the categories you are trying to distinguish between; i.e. normal and abnormal. Cases is the number of subjects or measurements. To see which features are stable between trials try intraclass correlation coefficients Nov 1, 2012 at 13:58
• I just realized I need to use ICC for this problem, thank you. This helps me a lot :)
– Neu
Nov 2, 2012 at 0:55

You may experiment with two representations of the problem.

1- Group ECG channels of one class together and label class A as 1 and class B as 2. So, the target variable is composed of two classes only. Now, you may apply some feature ranking methods which are independent of the classification process. For example, you have mRmR and JMI.

2- Every channel is a separate class and for each channel, there are two groups i.e. normal and abnormal singals. This can be viewed as a hierarchical solution. So, you start by classifying the signal into one of the 3 channels. After that, you progress by classifying the signal into A or B.

You need to pay attention that for different representations, the target variable differs and that may lead to different subsets of the selected features.

• Thank you, now I'm working with two different approach regarding the channels and I'll see if it'll affect the classifier's accuracy :)
– Neu
Nov 2, 2012 at 0:53