0
$\begingroup$

For my task, I am doing unsupervised learning and I am trying to find the best possible value of the parameters gamma and nu in OneClassSVM.

I did the following:

nus = [0.001, 0.01, 0.1, 1]
gammas = [0.001, 0.01, 0.1, 1]
tuned_parameters = {'kernel' : ['rbf'], 'gamma' : gammas, 'nu': nus}
tuned_ocsvm = OneClassSVM()
clf = GridSearchCV(tuned_ocsvm, tuned_parameters, cv=10)
clf.predict(maximum.values[:,[1]])

and I am getting, the following error:

NotFittedError: This GridSearchCV instance is not fitted yet. Call 'fit' with appropriate arguments before using this method.

The above error makes sense since the instance wasn't fitted which led me to to do this:

nus = [0.001, 0.01, 0.1, 1]
gammas = [0.001, 0.01, 0.1, 1]
scores = ['recall']
tuned_parameters = {'kernel' : ['rbf'], 'gamma' : gammas, 'nu': nus}
tuned_ocsvm = OneClassSVM()
clf = GridSearchCV(tuned_ocsvm, tuned_parameters, cv=10, scoring = scores, refit = False)
clf.fit(maximum.values[:,[1]])

and this gave the following error:

TypeError: __call__() missing 1 required positional argument: 'y_true'

But I don't have the 'true' values available since I am doing unsupervised learning.

Is there any other way the best parameters can be found for the same?

$\endgroup$
  • $\begingroup$ What unsupervised problem are you trying to tackle? Clustering? Outlier prediction? Density estimation? $\endgroup$ – deemel Apr 11 at 18:52
  • $\begingroup$ @Rickyfox: Both clustering and outliers $\endgroup$ – Junkrat Apr 11 at 21:17
0
$\begingroup$

Unsupervised learning, as commonly done in anomaly detection, does not mean that your evaluation has to be unsupervised. In order to evaluate different models and hyper-parameters choices you should have validation set (with labels), and to estimate the performance of your final model you should have a test set (with labels).

Usually the assumption is that all data in the training set is "normal" (not an anomaly). So you need to find (or create) some anomalies in your dataset. This can be done through Exploratory Data Analysis.

Should have least 20 anomaly instances, but something like 200-2000 is of course better. Then you remove those from the candidates for training data, and split out a validation and testset. A split train/validation/test ratio might be 80/10/10 or 60/20/20 for the the negative class (normal), and then 0/50/50 for the labeled anomalies (so if you have 20 labeled anomalies, 10 for validation and 10 for test).

Once you have this you can perform gridsearch. Though unfortunately scikit-learn GridSearch does not let you specify train and validation sets manually, but you an use ParameterGrid and implement it manually. I would definitely plot the scores across hyperparameters and chose manually, as with a small validation set the scores can be pretty noisy.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.