I understand intuitively why this is a horrible idea - you assume your model is correct and then increase your number of observations which will likely result in a poor fit on future data.
I'm wondering if there is some mathematical/statistical property to describe this, or if there is any rare case where this may not be as fatal as I am thinking?
I can give you a probabilistic/Bayesian interpretation of why this is not helpful. A probabilistic model for data $X$ and parameters $\theta$ is defined by a likelihood $P(X|\theta)$ and a prior $P(\theta)$. Now imagine I have some training data $X_\text{train}$ and want to make predictions about future data $X_\text{future}$, which means I need to calculate, or approximate $$ P(X_\text{future}|X_\text{train}) = \int P(X_\text{future}|\theta) P(\theta|X_\text{train}) d\theta $$ where $P(\theta|X_\text{train})$ is the posterior. What you suggest is to sample predictions $X_\text{pred}$ from $P(X_\text{pred}|X_\text{train})$ (which can be represented in the same way as the above equation). However, since $X_\text{pred}$ is not observed you can integrate it away and your posterior on $\theta$ will be unchanged. Conditioning on $X_\text{pred}$ is therefore not a reasonable thing to do.
To speculate on the typical effect it might have: if you sample from $P(X_\text{pred}|X_\text{train})$ then you are both adding noise to your estimate, and reducing the uncertainty in the estimate of $\theta$ (so you would probably be both overconfident and more wrong!), whereas if you optimise $P(X_\text{pred}|X_\text{train})$ I expect the main effect would be reducing the uncertainty in the posterior and thereby making your predictions overly confident (i.e. overfitting).
It's not bad and actually may improve quality quite a lot if you have a lot of unlabeled data (i.e. $X$'s without $Y$'s) and some labeled data: you train a model with labeled data, label unlabeled with your model and retrain the model. This falls into the class of so-called semi-supervised learning methods: http://en.wikipedia.org/wiki/Semi-supervised_learning
