As with many such situations, one must take care to avoid confusing sample and population quantities. (Given some particular distributional assumptions, we might choose to test for symmetry about a population mean using a statistic based on sample medians for example.)
We should also keep in mind that failure to reject a null of symmetry is not the same as showing symmetry.
Let's begin by simplifying things by assuming continuity.
First, what is meant by symmetry of a distribution? While it's usually conceived in the elementary treatments in terms of the density - i.e. as $f(\theta+x)=f(\theta-x)$, when we say 'that the distribution is symmetric', I often tend to conceive it in terms of the distribution function (though the distinction won't matter, generally).
Note that symmetry around the population mean implies symmetry about the population median, so we needn't distinguish them - if the mean exists, the two will be the same.
There are two cases to distinguish:
testing for symmetry about a specified location and
testing for symmetry about an unspecified location
Let's consider each in turn
- One example of a way to test for symmetry about a specified mean $\theta_0$ is to create a second sample, $Y=2\theta_0-X$ and compute a test statistic that measures discrepancy in the distributions of X and Y (such as a two-sample Kolmogorov-Smirnov statistic).
[I'm not certain the distribution of the test statistic under the null is still the same as for the KS test $-$ and I'm not going to try to work it out right now (but my guess is that it isn't) $-$ but in any case the distribution could easily be simulated for this circumstance, so it's not a huge issue.]
Note further that testing for symmetry about a known location may be reduced to testing for symmetry about 0 simply by subtracting the given location from all the observations. The test mentioned above would then be a test for symmetry about 0.
There are many other tests that could be used in this situation, such as a sign test (if the distribution is not symmetric about 0, there will typically tend to be an excess or deficit of positive signs, though counterexamples are certainly possible), or the signed rank test mentioned before. (They all act as a test of symmetry about the specified population mean)
- Some tests for symmetry about an unknown center. There are many of these; I'll mention just a few.
i) The triples test of Randles et al (1980)
This test is (IMO) intuitively appealing. It looks at sets of three observations, checking whether in each case the triple has the middle observation closer to the smaller (suggesting right skew) or larger (suggesting left skew) observation (the right skew case gets a score of 1/3, the left skew case gets -1/3 and anything else scores 0. Then the test statistic, $R$, is the average of the scores over all possible triples.
(This test is not distribution free, but with a consistent estimator of the variance of $R/\sqrt n$ it is asymptotically distribution free.)
Randles, Fligner, Policello and Wolfe (1980)
An Asymptotically Distribution-Free Test for Symmetry Versus Asymmetry
Journal of the American Statistical Association
Vol. 75, No. 369, Mar., pp. 168-172
ii) Gastwirth's (1971) modified sign test. Gastwirth considered a sign test about the sample mean. It's no longer distribution-free, but again, with a consistent estimator of the variance of an appropriately scaled statistic, it is asymptotically so. However, note that this test would have very low power against asymmetric distributions with $P(X>\mu) = 1/2$
Gastwirth, J.L. (1971)
On the Sign Test for Symmetry.
Journal of the American Statistical Association, 66, 821-828.
iii) Hotelling and Solomons test (1932) of the Pearson skewness (scaled mean-median). Gastwirths 1971 paper (mentioned above) gives an expression for the asymptotic variance of a suitably normalized statistic and this, too, is thereby asymptotically distribution free.
Hotelling, H. and L. M. Solomons (1932)
The Limits of a Measure of Skewness
Ann. Math. Statist. Vol 3, No. 2, 141-142.
On this test, also see here
Note that Gastwirth's test in (ii) is quite similar to the test you propose, with only the substitution of the sign test for the signed rank test. Your test would also not be distribution-free, but you should probably be able to find a consistent estimator of the variance of your statistic (appropriately standardized), and thus get an aymptotically distribution free test. (Alternatively, you might be able to come up with a bootstrap test based off such a statistic.)
A review of tests of symmetry can be found here. Also see this tech report