How are the scores computed with SelectKBest (sklearn) I was expecting the scores_ provided by SelectKBest() to be the result of the score_func (e.g. the value of the F statistics itself when score_func = f_classif, or the chi2 statistics when score_func = chi2).
But it is apparently not the case. I get
selector = SelectKBest(f_classif, k=2)
selector.fit(x_train, y_train)

pd.DataFrame({'variable': x_train.columns,
              'score_f_anova': selector.scores_})

_variable_  |_score_f_anova_
variable_1  |43.525376
variable_2  |28.432673

which is different from the F statistics I get from:
from scipy.stats import f_oneway
print(f_oneway(x_train['variable_1'], y_train)[0])
print(f_oneway(x_train['variable_2'], y_train)[0])

141.423192895752
90.45067843845831

Therefore:


*

*How is scores_ computed?

*Which of scores_ and pvalues_ should I use to rank the features by importance?

 A: f_classif and f_oneway produce the same results but differ in
implementation and use.
First, recall that 1-way ANOVA tests the null hypothesis that
samples in two or more classes have the same population mean. In your case,
I suppose y_train is an array-like categorical variable containing some
classes, and x_train is an array-like or dataframe containing features.
Then the right way to pass sample measurements for each class into
f_oneway is
f_oneway(x_train[y_train == 'class1'], x_train[y_train == 'class2'], # ...,
         x_train[y_train == 'classk'])

or more conveniently
f_oneway(*[x_train[y_train == c] for c in np.unique(y_train)])

f_classif and other scoring functions from scikit-learn require both
features and labels:
f_classif(x_train, y_train)

Example:
>>> import numpy as np
>>> from scipy.stats import f_oneway
>>> from sklearn.datasets import make_classification
>>> from sklearn.feature_selection import f_classif
>>> n_samples, n_features, n_classes = 1000, 4, 3
>>> X, y = make_classification(n_samples=n_samples, n_features=n_features,
...     n_classes=n_classes, n_informative=3, n_redundant=1,
...     scale=0.2, random_state=0)
>>> f_tuple = f_oneway(*[X[y == k] for k in range(n_classes)])
>>> f_stat, f_prob = f_classif(X, y)
>>> np.allclose(f_tuple.statistic, f_stat), np.allclose(f_tuple.pvalue, f_prob)
(True, True)
>>> f_stat.shape[0] == n_features
True

Finally, SelectKBest produces exactly the same results as the score_func
function provided and stores them in the scores_ attribute. So,
your selector's scores are correct and will be equal to an f_oneway's
output if you pass the arguments properly.

If you rank features manually, it is up to you whether to rely on scores or
p-values. But If you apply scikit-learn's feature selection techniques, it
depends on the implementation. SelectKBest and SelectPercentile
rank by scores, while SelectFpr, SelectFwe, or SelectFdr
by p-values. If p-values are supported by a scoring function, then you can use
the latter three models with this function and some p-value (alpha, the
highest p-value for features to keep).
Example:
>>> from sklearn.feature_selection import SelectFpr
>>> s = SelectFpr(score_func=f_classif, alpha=0.01)
>>> X_t = s.fit_transform(X, y)
>>> X_t.shape
(1000, 3) # Recall that one feature was uninformative (redundant)

A: I would like to add that you can use the following function to use a scipy stats function into GenericUnivariateSelect.
def sk_func_from_scipy(scipy_func):
    def sk_func(X,y):
        sc_vect = []
        pval_vect = []
        for i in range(X.shape[1]):
            result = scipy_func(*[X[y == c][:,i] for c in np.unique(y)])
            sc_vect.append(result[0])
            pval_vect.append(result[1])
        return (np.array(sc_vect), np.array(pval_vect))
        
    print(scipy_func)
    return sk_func 

Then you can use scipy functions like kruskal or mannwithneyu:
import scipy.stats
fs = GenericUnivariateSelect(score_func=sk_func_from_scipy(stats.kruskal), mode='k_best', param=5)
X_selected = fs.fit_transform(X, y)

and
 fs = GenericUnivariateSelect(score_func=sk_func_from_scipy(stats.mannwhitneyu), mode='fpr', param=1e-5)
X_selected = fs.fit_transform(X, y)

Note that
GenericUnivariateSelect(score_func=sk_func_from_scipy(f_oneway), mode='k_best', param=5)

should give you the same results as
fs = SelectKBest(score_func=f_classif, k=5)

