Does this strategy work or is it considered overfitting? Imagine I have X_train, y_train and X_test, y_test.
I was studying neural networks and so I came up with the following strategy, however I don't know whether it is considered overfitting or not.


*

*Create Neural Network number 1, call it NN1.
a.) Train NN1 on X_train and y_train.
b.) Test NN1 on X_test and y_test. Call the score test_score_1 and the respective predictions test_pred_1.
c.) Just for later reference, evaluate NN1 on training set, calling train_score_1 the score and the respective predictions train_pred_1.

*Create a secon Neural Network and call it NN2.
a.) Train NN2 with input X_train and train_pred_1 and with output y_train. (Basically the only difference is that there is a new feature. This new feature are the training predictions of NN1).
b.) Test NN2 on inputs X_test and test_pred_1, with output y_test. Keep the score as test_score_2 and the predictions as test_pred_2.
c.) Now evaluate NN2 on the training set, calling train_score_2 the score, and the respective predictions train_pred_2.
My questions


*

*Does this make sense (especially if we imagine to repeat this with multiple NN, say NN3, NN4,...)?

*Is this overfitting?

*If we compare test_score_1 and test_score_2, and we have that the second score is better (lower) than the first, does it mean we are not overfitting?

 A: 1) Congrats, you've invented boosting :D Typically a simpler model is used, such as decision trees or a linear model. 
2) It's not overfitting. You are not using y_test, nor X_test for the training of the networks (though X_test would be fine in some contexts, as a semi-supervised learning)
3) If you've trained, say, 100 networks, you have to use the 100th test predictions. Otherwise you are doing validation overfitting. You could use the validation set to determine what is the optimal number of models to ensemble. Typically, if the k-th model worsens your predictions, the residuals are just noise at that point. 
A: What you're describing is more akin to autoencoding than boosting.
Answers:
1) Possibly, and any potential benefits are likely to manifest themselves only in the case of high dimensional data (lots of predictors). A univariate response regression model simply maps $P$-dimensional predictor vectors into a $1$-dimensional response space. That is, the predictions are just the conditional expectation $E[Y]$ given parameters/weights $w$ and conditioned on predictor values $x$. This mapping (or transformation) always results in some information loss; otherwise, the original model would be perfect. So you're not providing the second model in the ensemble any additional information by compressing (to use a CS analogue) -- with information loss -- the information fed to the first model in the form of predictor vector $x$. There is evidence that this sort of compression can improve model performance under some circumstances -- see my Wikipedia link above.
2) The question of whether or not a model is overfit is not generally boiled down to a "yes" or "no" scenario; in this case, it's not immediately apparent that this approach would increase variance (higher variance is associated with overfitting). I think the main concern here is whether your modeling scenario is of such high dimensionality that compression is necessary; I'd worry about that before getting into the bias/variance stuff.
3) I wouldn't necessarily draw any conclusions regarding overfitting from the performance of NN2 vs NN1. There are many reasons why this sort of compression might make NN2 superior to NN1, and not all of them imply that compression is the best way to achieve that improvement. I think it's very likely that there are simpler ways unless you have huge dimensionality.
