22

Remember that your neural network outputs probabilities, not classifications. It is up to you to use the probabilities to make classifications. The software default is to use a threshold of $0.5$ to determine classification, but that might not be the right number for you. While there are two outcomes, there is likely a third option in the decision-making ...


9

A standard way to go about this is as follows: As mentioned in Dave's answer, instead of taking the binary predictions of the Keras classifier, use the scores or logits instead -- i.e. you need to have a confidence value for the positive class, instead of a hard prediction of "1" for the positive class and "0" for the negative class. (...


8

In the example of the video, people are classified as "Hipster" or "Non-Hipster". That is a nominal scale level. Nominal data can be counted but not added. Without addition, there is no computing means. The mean of a "Hipster" and a "Non-Hipster" is not "Non-Hipster and a half". You can count people and ...


2

You can do a one-sided, one-sample test of proportions. If $p$ represents the proportion of games where White wins, then your null hypothesis is that $p = 0.5$ (i.e. White has no better chance of winning than random chance, which is 50/50) and the alternative hypothesis would be that $p \geq 0.5$ (i.e. White has a greater than random chance of winning).


2

You can use logistic regression with an ordinal predictor variable. By choosing the encoding system for the predictor, you can get the information presented in a useful form. Here is a useful UCLA page overview for different categorical encoding systems, using R (there are similar pages for other languages.) For your purposes, maybe a successive differences ...


1

AUROC is a semi-proper scoring rules and actually uses the raw probabilities to calculate the best threshold to differentiate the two classes, that is in comparison to a default call to predict, which uses the "non-informative" threshold of 0.5. Other measures such as accuracy, F1, recall, and others are not proper scoring rules, and they work on ...


1

Choosing a threshold depends on what trade-off you wish to achieve in terms of classification errors. Choosing 0 or a different number could be appropriate, depending on context. The network is trained using BCEWithLogitsLoss, which combines the sigmoid activation and the binary cross-entropy loss into a single call. This is described in the pytorch ...


Only top voted, non community-wiki answers of a minimum length are eligible