I've been training on predicted labels and then using that to predict the train data, using the resulting score as a measure of fitness of my model.

It seems to work quite well in many different scenarios, though I do notice that some level of accuracy is required otherwise the score can be a bit noisy and less useful.

I'd like to learn more about this technique. Does it have a name or are they any references I can check out to better understand what I'm doing? I'd like to use it for epoch selection, forward feature selection, hill climbing, etc.

Note that this technique isn't really cross validation in the sense of oof / folding. It's more inverting the prediction process and a semi supervised learning coherence process.

Also note that I'm not proposing using predicted labels for training, just for measuring model fitness. I find that can lead to overfitting, though I suppose so can what I'm suggesting, at least indirectly.

example code:

model_to_be_evaluated = LGBMRegressor()
model_to_be_evaluated.fit(Xtrain, ytrain)
SSL_model_evaluator = LGBMRegressor()
SSL_model_evaluator.fit(Xtest,model_to_be_evaluated.predict(Xtest))
score_for_model_to_be_evaluated = SSL_model_evaluator.score(Xtrain, ytrain)

The idea being that I'm trying to blend in information regarding the unlabeled test data and how they interact with the predictions. I see significant correlation in many scenarios (not all) with CV score / test scores.

In some situations, all you have are predicted labels. The correlation is generally perfect against scores on ground truth at a coarse enough level in most scenarios, which makes it useful for some purposes.

At the very minimum, I find it very useful as a quick sanity check.

edit to add that this might help with overfitting (Bjorn's point):

SSL_model_evaluator.score(Xtrain_holdout, ytrain_holdout)