1
$\begingroup$

I am obtaining two very different accuracies for the AT&T face database when fitting the model with lda & qda. Before using QDA I first search for the ideal regularisation parameter, AFAIK the only import parameter to fine-tune for QDA.

X_load,y_load = datasets.fetch_olivetti_faces(data_home="R:/DATASET/AT&T", return_X_y=True)

I split this into a balanced train and test sequence (8 images for training, 2 for testing per person)

lda = LinearDiscriminantAnalysis(solver='svd')
lda.fit(X_train, y_train)
y_pred_lda = lda.predict(X_test)
y_true_lda = y_test
f1_scores_lda.append(met.f1_score(y_true_lda, y_pred_lda, average='micro'))

qda = QuadraticDiscriminantAnalysis()
clf = GridSearchCV(qda, params, cv=4)
clf.fit(X_train, y_train)
reg_params_qda.append(clf.best_params_['reg_param'])

Im running this experiment for an increasing number of persons so Im keeping a python list with these parameters

qda2 = QuadraticDiscriminantAnalysis(reg_param=clf.best_params_['reg_param'])
qda2.fit(X_train, y_train)
y_pred_qda = qda2.predict(X_test)
y_true_qda = y_test
f1_scores_qda.append(met.f1_score(y_true_qda, y_pred_qda, average='micro'))

When i run this using the whole dataset (40 persons);

f1_scores_lda outputs 0.975

f1_scores_qda outputs 0.125

When i run this for 10 persons;

f1_scores_lda outputs 0.9

f1_scores_qda outputs 0.3

Why is QDA performing so poorly?

I'm getting "Variables are collinear" warning for QDA, what can I do about this?

$\endgroup$
0
$\begingroup$

I figured it out, QDA needs to be trained on less features! (Images are 64x64 pixels => 4096 features)

LDA can be used for dimensionality reduction in a first step, keeping let's say 6 discriminating features.

In this subspace, you can train a QDA classifier and on the AT&T dataset it slightly outperforms LDA. (see picture below, for 5 fold crossvalidation using the entire dataset)

My mistake was that in a rush I trained it like I trained LDA, forgetting QDA can't do dimensionality reduction.

5 fold crossvalidation scores on AT&T

Some code to illustrate this;

lda = LinearDiscriminantAnalysis(solver='svd')
crossval_scores_lda.append(cross_val_score(lda, X_load, y_load, cv=5))

lda = LinearDiscriminantAnalysis(solver='svd', n_components=6)
X_load_qda = lda.fit_transform(X_load, y_load)
qda = QuadraticDiscriminantAnalysis()
clf = GridSearchCV(qda, params, cv=4)
clf.fit(X_load_qda, y_load)
reg_params_qda_after_LDA.append(clf.best_params_['reg_param'])
qda2 = QuadraticDiscriminantAnalysis(reg_param=clf.best_params_['reg_param'])
crossval_scores_qda.append(cross_val_score(qda2, X_load_qda, y_load, cv=5))
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.