While reading the neuroscience paper "Neural population dynamics during reaching" by Churchland et al. 2012, Nature, the authors mention using 'soft' normalization of their (biological) neurons firing rates such that 'neurons with very strong responses were reduced to approximately unity range, but neurons with weak responses had less than unity range'. I have never came across this kind of normalization, and the supplementary information does not give any details on this method. I am pretty sure that they are not 'softmax' normalization as I don't see how this would yields a unit range for some neurons. Any ideas ?
The equation they used was:
respose_normalized = response/(range(response) + 5)
The idea is to roughly normalize the activity of each unit to exist on a comparable scale, but to avoid giving low firing (and thus unreliable) units too much influence. The value of 5 was chosen as a somewhat arbitrary threshold for "low activity".