Prediction with scikit and an precomputed kernel (SVM) I am kind of a newbie in the MachineLearning area and evaluating some tools etc. to get a feeling for it.
For a project I am using a tool that creates a precomputed kernel (gram matrix) and also is able to normalize it (values between 0 and 1).
My problem is, that I am not able to predict anything other than the training-set when using the precomputed kernel.
Short summed up problem-description:


*

*original dataset is 1285 instances

*split it by using k_folds = sklearn.cross_validation.StratifiedKFold(list_of_annotations, k=5) into 5 folds (splitting manually, using the indices from the method)

*running (for test-purposes only on one fold for now)
`classifier = svm.SVC(kernel='precomputed', probability=True) 
clf = classifier.fit(train_matrix, train_annotations)
--> seems to work / no problems until here


*

*clf.predict(test_matrix) gives me the following error-message:




ValueError: X.shape[1] = 257 should be equal to 1028, the number of samples at training time


Is my approach wrong?
I don't really understand the error-message - if I train the SVM with the precomputed kernel, afaik I'll not always have the same shape of the input-data (precomputed kernel on the input-data) for prediction as in the training-set?!
 A: It seems that gram matrix that you use for predictions is wrong.
Once you fit the SVM its prediction for $x$ is: $ y = \text{sign} \langle w,\phi(x)\rangle $
It sure is possible that one cannot compute $\phi(x)$ but needs to use a pre-computed gram matrix instead. Substitute $ w = \sum_i^m\alpha_i\phi(x_i)$
into the above prediction expression:
$$
y = \text{sign } \langle \sum_i^m\alpha\phi(x_i),\phi(x) \rangle = \text{sign }  \sum_i^m\alpha_i\langle\phi(x_i),\phi(x) \rangle = \text{sign } \sum_i^m\alpha_i K(x_i,x) \rangle = \text{sign } (Xx)^T\alpha = \text{sign } {G}\alpha
$$
Where $X$ is the design matrix for the training data (row $i$ of $X$ is $\phi(x_i)$) and gram matrix $G=(Xx)^T$ (note that G is not symmetric)
You can use this as a reference:
from sklearn.datasets import load_digits
from sklearn.svm import SVC
from sklearn.utils import shuffle
from sklearn.metrics import accuracy_score
import numpy as np


digits = load_digits()
X, y = shuffle(digits.data, digits.target)
X_train, X_test = X[:1000, :], X[100:, :]
y_train, y_test = y[:1000], y[100:]

svc = SVC(kernel='precomputed')

kernel_train = np.dot(X_train, X_train.T)  # linear kernel

svc.fit(kernel_train, y_train)

#kernel_test = np.dot(X_test, X_train[svc.support_, :].T)
kernel_test = np.dot(X_test, X_train.T)
y_pred = svc.predict(kernel_test)
print 'accuracy score: %0.3f' % accuracy_score(y_test, y_pred)

A: I find that I usually get that error from training/vectorizing outside the kfold loop.
vec.fit_transform(), vec.fit_transform(), clf.fit(), clf.pred() - all four of those function calls has to be inside to loop, to make sure all matricies gets the correct shapes.
Is your testing algo setup something like this?:
k_folds = sklearn.cross_validation.StratifiedKFold(list_of_annotations, k=5)

for train_index, test_index in k_folds:
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]

    vec = TfidfVectorizer()
--> X_train_matrix = vec.fit_transform(X_train)
--> X_test_matrix = vec.transform(X_test)

    clf = svm.SVC()
--> clf.fit(X_train_matrix, y_train)

--> y_pred = clf.predict(X_test_matrix)
    precision, recall, fscore, support = precision_recall_fscore_support(y_test, y_pred)
    # print/save accuracy scores

