I am testing if the sound pressure levels(rms) of shipping noise affect the presence or absence of the fin whales by acoustic monitoring for 4 months of data from one location. Here my response variable is binary with 0 for the absence of acoustic detection and 1 for the presence and the main predictor is the variable "rms" is continuous(dB re 1microPa), along with some environmental variables. My null hypothesis that the shipping noise does not affect the presence of Fin whales detections and my final logistic regression model after completely a forward selection is : glm(formula = FWbi ~ rms + shptnl + day + hour + partday + month + ssh + chlora, family = binomial, data = noise).
I conducted k-fold cross-validation and the confusion matrix is as follows.
crossValSettings_fw <- trainControl(method = "repeatedcv", number = 10, savePredictions = TRUE) crossVal_fw <- train(FWbi ~ rms + shptnl + day + hour + partday+ month + ssh + chlora ,family = binomial,data = noise, method = "glm", trControl= crossValSettings_fw) pred <- predict(crossVal_fw, newdata = noise) confusionMatrix(data = pred, noise$FWbi)
The dataset/Reference of the actual absence of fin whale and Prediction correctly predicts the absence of fin whale(true positive)- 102078. The dataset/Reference the actual presence but Prediction falsely predict it as the absence of fin whale(false positive)- 4447. The reference of the actual absence of fin whales but the prediction falsely predicts the presence of fin whale(false negatives) -19. The reference of the actual presence of fin whales and the prediction correctly predicts the presence of fin whales(true negative)- 15.
Accuracy : 0.9581 95% CI : (0.9569, 0.9593) No Information Rate : 0.9581 P-Value [Acc > NIR] : 0.5284 Kappa : 0.006
Mcnemar's Test P-Value : <2e-16
Sensitivity : 0.999814 Specificity : 0.003362 Pos Pred Value : 0.958254 Neg Pred Value : 0.441176 Prevalence : 0.958126 Detection Rate : 0.957948
Detection Prevalence : 0.999681
Balanced Accuracy : 0.501588
'Positive' Class : 0
AUC@y.values 2 0.8425831
AUCrms@y.values 2 0.6886156
According to my interpretation of this, it can be seen that my false positive is higher than my true negative and seems to cause a type 1 error according to what I know. I would really appreciate if someone could tell me if I interpreted it right or not. If yes, can I fix type 1 errors? And how do I fix it? Also, my data is zero heavy so idk if that is causing the problem? I looked up several sites but couldn't conclude on how to end my dilemma. Please help.