Binary classification: does it make a difference to use zero or minus one as class label? I'd like to build a binary classification model and I recall reading somewhere that the choice of the labels could have an impact depending on the algorithm. So the two modeling approaches are to either:


*

*Use class labels [0, 1] as levels for the response variable.

*Use class labels [-1, 1] as levels for the response variable.


The question is: does it make a difference for certain models e.g. SVM to use 0 instead of -1 as response variable class label or level? If so can you please also provide references where this is explained? 
 A: Yes, it all depends on the algorithm and the cost function. For example, logistic regression's output is between 0-1 and you can't calculate cross entropy loss with labels -1 and 1. Similar things happen in neural nets, you'd want to use tanh in the last layer (because it's output is between -1 and 1). But, you won't be able to use the cross entropy loss function again. In SVM, usage of -1,1 is more common due to the problem formulation. In contrast to the comment under your question, hinge loss get along well with signed labels.
A: I disagree somewhat with gunes.
For all practical reasons (e.g. code implementations) the representations matter (if the package code doesn't transform the representation to what it needs). They are also "better" in the sense of simpler. But they aren't essential.
The choice is simply a smart representation to facilitate the algorithm. It's a computation "trick", or a "shortcut".
In SVM, the objective is reduced to the following:
$$ \begin{array}{ll} 
  \text{min} & ||w||^2\\ 
  \text{subject to} & y_i (w^Tx_i+\beta)\ge 1-\xi_i, \forall i=1...n\\ 
 \end{array}$$
But there's nothing to prevent you from doing:
$$\begin{array}{ll} 
\text{subject to} & w^Tx_i+\beta\ge 1-\xi_i, \forall i\in \text{Class A} \\
&w^Tx_i+\beta\le -1+\xi_i, \forall i\in \text{Class B}\end{array}$$
And give what-ever label you want to class A or B. It just makes the computation more complex and less elegant.
In Logistic Regression (ML-style, i.e. cross entropy + gradient descent), or any other algorithm that outputs a probability $p$ of being in a certain correct class, the Cross-Entropy loss wants to give a penalty of $\log(p)$ to how much you were wrong. Obviously since this is the correct class, $p$ should have been $1$. So the loss would be $0$ if you are correct, and can reach up to $-\infty$ if you assign $\to 0$ to the probability $p$.
For binary classification it is then easy to choose $0$ or $1$, since you can write the loss as $$-y_i \log{\hat y_i} - (1-y_i)\log{(1-\hat y_i)},$$ i.e. one expression for both the classes! But it isn't essential for it to be this way. This is not part of the definition of Cross-Entropy. In multiclass classification, which also uses Cross-Entropy, it's easier to represent the response as a 1-hot-vector, because then you can compute the element-wise multiplication of $\boldsymbol y$ with the vector of log probabilities $\log \boldsymbol p$.
Nothing prevents you to implement an "if" or a "switch" algorithm that checks which class your current observation is in, and calculating the loss for it. But it is less elegant and (subjectively) more complex.
For Logistic Regression GLM-style (Solving the Likelihood equations using Newton-Method/Fischer-Scoring/IRLS), the probability model is a Bernoulli distribution, and this is a bit more "hard-wired" to the whole modeling of the problem, so 0-1 representation is probably a must.
