I am a non-mathematical R programmer who is completely new to the idea of scoring rules. I would like to start using them instead of classification evaluation measures like accuracy and recall, which I have recently learned are improper in terms of scoring rules.
As a programmer, I work with statistics entirely from an input-output perspective. On the input side, I study to understand the necessary input parameters and options for a statistical procedure and, crucially, the assumptions under which the procedure is appropriate, including the necessary paramter adjustments under various circumstances. On the output side, I study to understand the meaning of the results provided, how to interpret them, and crucially, how to translate these results into real-world decision making. I do not have the mathematical foundations to understand the internal workings of the procedures, so I mostly gloss over such explanations. This approach serves me quite well.
I think I do understand the general notion: in my lay terms, I understand a strictly proper scoring rules to be a mathematical function that receives two key inputs: a set of probabilities of event outcomes; and a set of actual occurrences of the predicted events. A strictly proper scoring rule returns as output a score. When comparing the scores of two different forecasters, a strictly proper scoring rule always gives the better score (which could be high or low, depending on the scoring rule) to the forecaster that is more correct than the other.
Although there are existing R packages that implement scoring rules (https://cran.r-project.org/web/packages/scoringRules/index.html and https://cran.r-project.org/web/packages/scoring/index.html), I am unable to find any non-mathematical tutorial on how to use them meaningfully. For an example of the kind of thing I am looking for, here is a resource from which I learnt ordinal regression in R: Ordinal Logistic Regression | R Data Analysis Examples.
My question is: is there any such resource that I could use to learn how to implement and interpret scoring rules in R?