is it valid to use pearson correlation in order to create new features? I'm relatively new to machine learning and I was wondering if I can use the correlation between features and the target value to manipulate the features.
i'm trying to do some manual feature construction by looking at correlation of each feature with target value, like it is used for feature selection but here i want to use this information to create a new feature, for example we have two features 'a' and 'b', by looking at the correlation we see that their correlation is a = 0.1 and b = 0.3, now am i allowed to give the something like a score like a = 1 and b = 2 and then create a new feature c = a+b.
my concern is that i'm looking at the target value to do this.
Is it valid? is it considered falling in the trap of looking at the target value?
if it is valid it should be done only on the training set, right?
 A: I think that you're thinking about a clustering-type process, where you extract important subsets of the input space. Usually this is represented with finding shapes or patterns in the data to exploit. These patterns might not always emerge just looking at descriptive statistics. If you want to cluster the input based on purely observed inputs, the principle components analysis procedure can help you construct new variables that are linear combinations of the original input dimensions. If you want to incorporate the output in the clustering approach, you could use an active subspace linear or quadratic model to simillarly identify important directions in your input space.
A: In general, using target values isn't a very sound procedure. I'm guessing you'll have to rely on an unsupervised method to generate target values for your test data. So you can imagine that your solution will depend on the strength of your unsupervised method, which is obviously sub-par, otherwise you would have just used the unsupervised method in the first place. 
However, in practice, it's something that people do all the time. For example, speech recognition strongly relies on an estimate of the boundaries of phonetic units for its classification. The boundaries are then updated and fed back into the classifier for another iteration. This procedure usually converges to something. 
In conclusion, it's alright if you decide to incorporate target values in your features. However, be aware that your system will need target values in the test stage. In that case you'll probably end up with an iterative procedure of generating targets and updating them to get "better" target values. 
