0
$\begingroup$

I'm testing a dataset for various types of regression, comparing test error for each one to the Mean Prediction Error, that I found at the beginning. Unfortunately I don't have any experience in this field yet so I wanted to ask you if I did things well cause I get really high Test Error, even if it is lower than Mean Prediction Error, as it should be, shouldn'it? Thak you in advance, here is my MATLAB code:

clear close all

varnames= {'LongPos'; 'PrismaticCoef'; 'LengDispRatio'; 'BeamDraughtRatio'; 'LengthBeamRatio'; 'FroudeNumber'; 'ResResistance'};

YachtHydroDynamics = readtable('yacht_hydrodynamics.csv','Delimiter',' ');

YachtHydroDynamics.Properties.VariableNames = varnames;
% 
 YachtHydroDynamics(any(ismissing(YachtHydroDynamics),2), :) = [];

X=table2array(YachtHydroDynamics(:,1:6));
Y=table2array(YachtHydroDynamics(:,7));

n=size(X,1);

meanX=mean(X);
stdX=std(X);

Xp=(X-repmat(meanX,n,1))./repmat(stdX,n,1);

Xtrain = Xp(1:192,:); Ytrain= Y(1:192,:);
Xtest = Xp(192:end,:); Ytest = Y(192:end,:);

%OLS
Xtable = table(Xtrain(:,1),Xtrain(:,2),Xtrain(:,3),Xtrain(:,4),Xtrain(:,5),Xtrain(:,6),Ytrain,'VariableNames',varnames);
lm1 = fitlm(Xtable,'ResResistance~ LongPos+PrismaticCoef+LengDispRatio+BeamDraughtRatio+LengthBeamRatio+FroudeNumber');

mm = (Ytest-mean(Ytest)).^2;
MeanError=sum(mm)/60

[predYtest, predYtestCI] = predict(lm1,Xtest);
pp=(predYtest-Ytest).^2;
TEST_ERROR_LS=sum(pp)/60  
std_TEST_ERROR_LS=sqrt(var(pp)/60); 

%STEPWISE
steplm=stepwiselm(Xtable,'Criterion','bic');

[predYtest, predYtestCI] = predict(steplm,Xtest);
pp_step=(predYtest-Ytest).^2;
TEST_ERROR_STEPWISE=sum(pp_step)/60
std_TEST_ERROR_stepwise=sqrt(var(pp_step)/60);

%LASSO
[B,Lasso] = lasso(Xtrain,Ytrain,'CV',10);
betahat_lasso=lasso(Xtrain,Ytrain,'lambda',0.4936);
predYtest=mean(Ytrain)+Xtest*betahat_lasso;
pp_lasso=(predYtest-Ytest).^2;
TEST_ERROR_lasso=sum(pp_lasso)/60
std_TEST_ERROR_lasso=sqrt(var(pp)/60);

%PCR
v=pca(Xtrain);
Z=Xtrain*v(:,1:5);
lmZ=fitlm(Z,Ytrain);
beta_PCR=v(:,1:5)*lmZ.Coefficients.Estimate(2:end);
predY_PCR=mean(Ytrain)+Xtest*beta_PCR; 
pp_pcr=(predY_PCR-Ytest).^2;
TEST_ERROR_PCR=sum(pp)/60

As Mean prediction Error I get something like 275 and for the various techniques I get something between 87-89.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.