# Suggestions required from experts for performance improvement for a binary classification problem using timing data

I am a currently working on location verification using machine learning and neural network techniques. This is a classification problem where the system has to classify whether a user (based on his reported location) is genuine or malicious. The working of the designed system is based on Time of Arrival (ToA) of the user's transmitted signal.

• A user transmits a signal carrying his claimed location.
• Multiple base stations (BSs) independently 'measure' the ToA of the user's transmitted signal. Each BS also 'calculate' a second ToA using the user's claimed location and its (BS) own location.
• So each BS has two readings (features); a measured ToA and a calculated ToA.
• The 'measured' ToA has added non line of sight time biases in them.
• The training and test data has genuine (whose true and claimed location are the same) and malicious (whose true and claimed location are different) users.

I have used feedforwrad neural network with different architectures and found that a neural network with an input, a hidden (10 neurons) and an output layer (with binary output) produces better accuracy in classifying the users. I used ReLu, and linear transfer functions in the hidden, and output layers respectively.

Recently, in a hunt for performance improvement, I adopted SVM approach to the same problem and achieved slight improvement in classifying the users but the results are still not promising.

Can any expert here guide on suggesting changes to the discussed feedforward neural network architecture and/or SVM method that can possibly lead to performance improvements. I would welcome suggestions related to adopting an alternative approach other then the approaches discussed here that can lead to better classification outcomes (using timing data).

% Matlab SVM code

rng(0) % to reproduce the results.

data = data(randperm(end),:); % randomize the data

% Data has 5000 examples. 50% for genuine users and 50% for malicious users.

X = data(1:5000,1:end-1); % ToA inputs

y = data(1:5000,end); % Target

% Dividing the data into train and test set.

rand_num = randperm(5000);

X_train = X(rand_num(1:4000),:);

y_train = y(rand_num(1:4000),:);

X_test = X(rand_num(4001:5000),:);

y_test = y(rand_num(4001:5000),:);

c = cvpartition(y_train,'k',5);

opts = statset('display','iter');

fun = @(train_data,train_labels,test_data,test_labels)... sum(predict(fitcsvm(train_data,train_labels,'KernelFunction','rbf'),test_data) ~= test_labels);

[fs,history] = sequentialfs(fun,X_train,y_train,'cv',c,'options',opts,'nfeatures',12);

X_train_w_best_features = X_train(:,fs);

Mdl = fitcsvm(X_train_w_best_features,y_train,'KernelFunction','rbf',... 'OptimizeHyperparameters','auto',... 'HyperparameterOptimizationOptions',struct('AcquisitionFunctionName',... 'expected-improvement-plus','Showplots',true));

X_test_w_best_features = X_test(:,fs);

accuracy = sum(predict(Mdl,X_test_w_best_features) == y_test)/length(y_test),... *100;