# How can I fix type 1 error for my logistic regression model?

I am testing if the sound pressure levels(rms) of shipping noise affect the presence or absence of the fin whales by acoustic monitoring for 4 months of data from one location. Here my response variable is binary with 0 for the absence of acoustic detection and 1 for the presence and the main predictor is the variable "rms" is continuous(dB re 1microPa), along with some environmental variables. My null hypothesis that the shipping noise does not affect the presence of Fin whales detections and my final logistic regression model after completely a forward selection is : glm(formula = FWbi ~ rms + shptnl + day + hour + partday + month + ssh + chlora, family = binomial, data = noise).

I conducted k-fold cross-validation and the confusion matrix is as follows.

crossValSettings_fw <- trainControl(method = "repeatedcv", number = 10, savePredictions = TRUE) crossVal_fw <- train(FWbi ~ rms + shptnl + day + hour + partday+ month + ssh + chlora ,family = binomial,data = noise, method = "glm", trControl= crossValSettings_fw) pred <- predict(crossVal_fw, newdata = noise) confusionMatrix(data = pred, noise\$FWbi)

Confusion Matrix and Statistics

The dataset/Reference of the actual absence of fin whale and Prediction correctly predicts the absence of fin whale(true positive)- 102078. The dataset/Reference the actual presence but Prediction falsely predict it as the absence of fin whale(false positive)- 4447. The reference of the actual absence of fin whales but the prediction falsely predicts the presence of fin whale(false negatives) -19. The reference of the actual presence of fin whales and the prediction correctly predicts the presence of fin whales(true negative)- 15.

           Accuracy : 0.9581
95% CI : (0.9569, 0.9593)
No Information Rate : 0.9581
P-Value [Acc > NIR] : 0.5284

Kappa : 0.006


Mcnemar's Test P-Value : <2e-16

        Sensitivity : 0.999814
Specificity : 0.003362
Pos Pred Value : 0.958254
Neg Pred Value : 0.441176
Prevalence : 0.958126
Detection Rate : 0.957948


Detection Prevalence : 0.999681
Balanced Accuracy : 0.501588

   'Positive' Class : 0


AUC@y.values 2 0.8425831

AUCrms@y.values 2 0.6886156

According to my interpretation of this, it can be seen that my false positive is higher than my true negative and seems to cause a type 1 error according to what I know. I would really appreciate if someone could tell me if I interpreted it right or not. If yes, can I fix type 1 errors? And how do I fix it? Also, my data is zero heavy so idk if that is causing the problem? I looked up several sites but couldn't conclude on how to end my dilemma. Please help.

• Can you include the confusion matrix instead of describing your results in words. Seems like you have a imbalanced dataset. Is there any variable that is associated / correlated with the presence in the first place? Apr 27 '20 at 17:07
• One way to manipulate Type 1 error rates in logistic regression prediction is to adjust the scoring threshold for classifying predictions into yes-no buckets. For instance, the usual rule-of-thumb is to use a threshold of the median score value, greater than that are yes, otherwise no.
– user234562
Apr 27 '20 at 17:17
• @StupidWolf Hey I have attached the picture for the confusion matrix. No, I ran correlation tests and vifs to remove correlated variables. My final model does not have any variables that are associated. Apr 27 '20 at 20:12
• @user332577 Hey thanks for the suggestion. I am very new to R and stats. Can you send me a link or the concept in R that I would need to use to do the scoring threshold for classifying? Apr 27 '20 at 20:17
• ok this is a bit tricky. You have mostly negative labels, so when you fit a logistic the intercept is going to be really low.. May not be easier for you to get a prediction > 0.5 . Can you try plotting your prediction probabilities against the actual label Apr 27 '20 at 21:03