Given that you used the term leave-one-out, I think what is implied by the question is that you want to do a LOO cross-validation. That is, you want to train on n-1 examples and test your model on the remaining example. In that case, being very methodologically strict, you are correct. The standardization is performed only on the training set, in this case, the n-1 examples. The for the test example, you subtract the mean of the training set and divide by the standard deviation of the training set. This will true for any cross-validation you choose. Standardization is part of the model you are creating, and you learn the standardization using only the training set, as you learn any other parameters using only the training set. And you apply the model on the test set.
In practice, almost never you need to be so strict. You can learn the standardization parameters (mean and standard deviation) on the whole set, and only then use the cross-validation. What is the consequence of doing this? What is happening is that once you define the cross-validation, you are using some information of the test set (its contribution to the general mean and standard deviation) in the training. That will usually result in a cross-validation measurement that is somewhat optimistic. If you are measuring X and the higher the X the better, using some information of the test set in learning step will result in a higher X measured on the test set that the "true" one.
If your final goal is to report the X itself, then your measurement will be a little bit inflated. This will be worse if you have few data points. But if your goal is to use the measure X to select among different models, then most likely the inflated X will be "the same" for all the models, and the order will be preserved, or, at least, most likely the best model will be the same in both cases. Again this will be more likely the more data you have.
To summarize, if you have a lot of data, do not worry, perform the standardization on the whole data set. If you have little data (and that will be a good reason to use LOO instead of other cross-validation schemes) then do perform the standardization as you described.