# How to compute the generalization error (test error) in a multivariate regression?

How can I, after running multivariate linear regression, compute the Generalization Error (also known as Test Error)? Are there many practises to do that?

EDIT: Can the following be considered as Generalization (or Test) Error?

y = data1; [M,N] = size(y);
X = [ones(M,N) data2];
y_train = y(M-30-483:M-30,:);
X_train = X(M-30-483:M-30,:);
y_test = y(M-29:M,:);
X_test = X(M-29:M,:);

[b,~,r,~,stats] = regress(y_train,X_train);
[M,N] = size(data2);
figure; plot(r);
axis([0,N+1,-Inf,Inf]);

y_estimated = X_test * b; y_expected = y_test;
generalization_error = y_expected - y_estimated;
figure; plot(generalization_error);

-
I take it that you're wondering how to do this in Matlab? –  MånsT Jul 22 '12 at 8:37
@MånsT: Basically yes, but I would also appreciate if you could provide a few hints regarding the theoretical aspects of this concept... –  user2295350 Jul 22 '12 at 19:51
possible duplicate you can check this stats.stackexchange.com/questions/32748/… –  s.s.o Feb 8 at 1:43

What you are showing above is indeed an estimate of the model performance on unseen data (generalization error). However the variance of your test data set will presumably be non-negligible and so a better way to estimate the generalization error would be through cross-validation.

Try the crossval function in Matlab or use cvpartition to define and your cross-validation folds and iterate though using a loop.

-
• root mean squared error sqrt (mean ((yhat - y)^2))
• mean absolute error mean (abs (yhat - y))