I'm learning how to do feature Engineering and come across some ideas in my head that's why I want to ask if I had some dataset with some features let's say 2 features and I have a timestamp column and the dataset is a time series dataset so they are monotonic. would it make sense to calculate the derivative or integral and add it as new feature ?

as an example let's say I have speed and acceleration as features, would it make sense to add the jerk (which is the derivative of the acceleration) and the snap(which is the derivative of the jerk) as new features ? also maybe the integral of the speed which would give the displacement I think?

the goal is let's say 2 features are not enough and we want to produce more features, is it wise to add the derivative or integral as a new feature? or is it a bad idea?

I also want to know whether the correlation between the derivative and integral according to timestamp and the feature that I derivated from would be high if I do this and is it bad or good if I make new features that correlate with others in my dataset


Why not? We even sometimes blindly include powers (and products) of independent variables to see if they contribute or not. It maybe the case that your target even directly is related to these derivative and/or integral (e.g. you have speed, but want to predict the force, i.e. $F=ma$). Depending on its properties, your algorithm may not find this by itself. It's best to try it out, and maybe afterwards employ some of the feature selection methods to see if they prevail.

| cite | improve this answer | |
  • $\begingroup$ thanks for your answer, yes your answer seems logic but I also read that when the features highly correlated that will make the model worse (at least that's what I read about neural networks) how right is this opinion? and is it true only for neural nets and not for other regression techniques like Random forest or Gradient boosting? $\endgroup$ – basilisk Dec 11 '19 at 19:05
  • $\begingroup$ Correlation doesn't mean any kind of relation, e.g. $x$ and $x^2$ can have zero-correlation (pearson) in some data ranges. Linear dependence (that may be the cause of high correlation) between features can sometimes be problematic. Linear regression is one example. NNs can also suffer from that. Tree algorithms typically deal with it. $\endgroup$ – gunes Dec 11 '19 at 19:09
  • $\begingroup$ please take a look here, there are many different opinion to this and it's making some confusion datascience.stackexchange.com/questions/24452/… $\endgroup$ – basilisk Dec 12 '19 at 12:04
  • $\begingroup$ correlation is not your case. Its differencing or cumulative summing. As I've indicated before, two variables being related somehow doesn't mean they're correlated. $\endgroup$ – gunes Dec 12 '19 at 19:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.