Should you ever standardise binary variables? I have a data set with a set of features. Some of them are binary $(1=$ active or fired, $0=$ inactive or dormant), and the rest are real valued, e.g. $4564.342$.
I want to feed this data to a machine learning algorithm, so I $z$-score all the real-valued features. I get them between ranges $3$ and $-2$ approximately. Now the binary values are also $z$-scored, therefore the zeros become $-0.222$ and the ones become $0.5555$.
Does standardising binary variables like this make sense?
 A: One nice example where it can be useful to standardize in a slightly different way is given in section 4.2
of Gelman and Hill (http://www.stat.columbia.edu/~gelman/arm/). This is mostly when the interpretation of the coefficients is of interest, and perhaps when there are not many predictors.
There, they standardize a binary variable (with equal proportion of 0 and 1) by
$$
\frac{x-\mu_x}{2\sigma_x},
$$
instead of the normal $\sigma$. Then these standardized coefficients take on values $\pm 0.5 $ and then the coefficients reflect comparisons between $x=0$ and $x=1$ directly. If scaled by $\sigma$ instead then the coefficient would correspond to half the difference between the possible values of $x$.
A: A binary variable with values 0, 1 can (usually) be scaled to (value - mean) / SD, which is presumably your z-score.
The most obvious constraint on that is that if you happen to get all zeros or all ones then plugging in SD blindly would mean that the z-score is undefined. There is a case for assigning zero too in so far as value - mean is identically zero. But many statistical things won't make much sense if a variable is really a constant. More generally, however, if the SD is small, there is more risk that scores are unstable and/or not well determined.
A problem over giving a better answer to your question is precisely what "machine learning algorithm" you are considering. It sounds as if it's an algorithm that combines data for several variables, and so it usually will make sense to supply them on similar scales.
(LATER) As the original poster adds comments one by one, their question is morphing. I still consider that (value - mean) / SD makes sense (i.e. is not nonsensical) for binary variables so long as the SD is positive. However, logistic regression was later named as the application and for this there is no theoretical or practical gain (and indeed some loss of simplicity) to anything other than feeding in binary variables as 0, 1. Your software should be able to cope well with that; if not, abandon that software in favour of a program that can. In terms of the title question: can, yes; should, no.
A: Standardizing binary variables does not make any sense.  The values are arbitrary; they don't mean anything in and of themselves.  There may be a rationale for choosing some values like 0 & 1, with respect to numerical stability issues, but that's it.  
A: What do you want to standardize, a binary random variable, or a proportion?
It makes no sense to standardize a binary random variable. A random variable is a function that assigns a real value to an event $Y:S\rightarrow \mathbb{R} $.  In this case 0 for failure and 1 to success, i.e. $Y\in \lbrace 0,1\rbrace$.
In the case of a proportion, this is not a binary random variable, this is a continuous variable $X\in[0,1]$, $x\in \mathbb{R}^+$.
A: In logistic regression binary variables may be standardise for combining them with continuos vars when you want to give to all of them a non informative prior such as N~(0,5) or Cauchy~(0,5).
The standardisation is adviced to be as follows:
Take the total count and give
1 = proportion of 1's
0 = 1 - proportion of 1's.
-----
Edit: Actually I was not right at all, it is not an standardisation but a shifting to be centered at 0 and differ by 1 in the lower and upper condition, lets say that a population is 30% with company A and 70% other, we can define centered "Company A" variable to take on the values -0.3 and 0.7.
