What exactly is the mathematical definition of a classifier / classification algorithm? I just started an intro machine learning course, and to get things better organized in my head, I was trying to come up with exactly what is needed to completely specify a classification algorithm. I understand a precise mathematical definition may not be possible. Here's what I have:
Let $\{X, Y\}$ denoted the data set (sample, label), $\theta$ the parameters.
A classification algorithm is a decision function $f(x; \theta)$ together with a cost/risk function $C(f(\cdot; \theta), X, Y)$. Specifying the functional form of $f$ and $C$ completely specifies the classification algorithm. $\hat{\theta} = \arg\min_\theta C(\theta)$ defines the classifier. 
Examples I'm thinking of are maybe logistic regression, where $f$ is the logistic function and $C$ is the cross entropy cost, or linear regression, where $f$ is the linear function and $C$ is the sum of squares cost, or the perception algorithm, where the function $f$ is the linear function and $C$ is the sum of individual losses consisting of the kinked function.
Is the above correct? Roughly? Where is it wrong?
 A: A classifier is a method that maps from inputs x to outputs l , where l are instances of a set of labels L.
There are many methods to build a classifier, an approach is:
define a variable y with value 1 when  l is a label l' and 0 when is not that label.
In this way, we can translate the mapping in estimating a function f(x;θ ) such that y=f(x;θ ) where f is user defined and the parameters  θ are selected to meet user stated  requirements ( for example low classification errors , few parameters,....).
Logistic Regression is an example, using this method allows to leverage on theory, algorithms, ..
A: A classifier algorithm is an algorithm that computes a classification based on some given input.
You describe more specifically the estimation of parameters $\hat\theta$ by minimizing some cost function. But the result of that is just one of many different classifiers.
The cost function does not define the classifier algorithm. And a classifier does not always need to be found by minimisation of a cost function.
A: My comments on your definition would be:
1) {X, Y} should be a pair, not a set (otherwise your clarification of which is which can't be computed). Check out kurtowski pair for background on how to define pairs using sets.
2) The classification algorithm (function) can't just know the parameters theta. It also needs to know the model form, or network architecture. The way you "specify the functional form" is by passing in the model as an argument.
I've just written out a post myself where I give my construction. It shows in detail how you functionally specify a model form. https://ndutoitblog.wordpress.com/2018/04/01/defining-machine-learning-with-maths/
Otherwise, I think you're pretty close. Good job.
