I sense this is not a good idea
I'd agree that it's not a good idea, if it can be avoided, but sometimes you only have one and that's all you can have.
but am struggling to think of an reason why it is wrong
'not a good idea' is not the same as 'wrong'. You're left with low power and an uncheckable assumption, both of which should be avoided if you can reasonably do so. But if the usual assumptions hold, it's all still valid.
other than that the assumption of homogeneity of variances would surely be violated.
You have no obvious basis to form an estimate of the population variance in the singleton group aside from the usual assumption of homogeneity of variance, but that doesn't mean it has been violated. On what basis do you assert that since n=1, homogeneity of variance is surely violated (aside from some basis that would apply to say the n=2 or n=5 case)?
Does it (the appropriateness of doing a one-way ANOVA under the conditions described) depend on how many participants are in the other levels of the factor, or on how many other levels there are?
No, it's still appropriate if the assumptions are satisfied, whether the other sample sizes are small or large, and whether there are only one other group or many. If you do it correctly, significance levels will still be as they should be. Power - of course - will be affected by changes in sample size.