Timeline for Why would you perform transformations over polynomial regression?
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 21, 2022 at 22:14 | comment | added | Dikran Marsupial | @dipetkov Domain knowledge isn't better when it is wrong. Some years back I helped organise a machine learning competition. In the first run, the datasets were given out without explanation of what the data meant (an "agnostic" setting) and in the second run, the same datasets were used with explanation of what the features were and the nature of the application ("prior knowledge" setting). IIRC the "agnostic" results were slightly better. | |
| Mar 21, 2022 at 8:46 | comment | added | dipetkov | @Igor This situation covers most of the social sciences. Just because there are no simple relationships like y = log(x), or my favorite so far y = exp{c + dx}, it doesn't mean that there is no expertise. It's just hard to put in a formula. And sometimes there are conflicting opinions. I wouldn't dismiss this as "these are no experts". | |
| Mar 21, 2022 at 8:36 | comment | added | Igor F. | @dipetkov Yes, but... in that case I'd say that even the "experts" lack the domain knowledge. | |
| Mar 21, 2022 at 8:35 | comment | added | Igor F. | @whuber My third point refers to the physical range of possibilities. I believe this is clear from the consecutive paragraph, referring to $1/x$. Regarding the 1st point, I guess it's useful to distinguish between the physical model and the errors. In the above case, an asymptotically falling, always positive curve is likely a reasonable physical model, but I see that the log-transformation of the fuel efficiency might fail due to negative observed values. | |
| Mar 20, 2022 at 16:25 | comment | added | dipetkov | Domain knowledge is valuable but always better? This example is from mechanics, so presumably there are physical laws that determine mpg and they can be reasonably approximated as a linear function of the measured predictors. In other domains it might be a lot less obvious how to express knowledge in terms of simple mathematical functions. Maybe sometimes deferring to experts would be a way to introduce bias. | |
| Mar 20, 2022 at 14:15 | comment | added | whuber♦ | The third point is not usually relevant: it makes little sense to insist that a model be accurate for the entire mathematical range of possibilities. Such a requirement would rule out the use of any principle of Newtonian physics, for instance. The first point is helpful but ought to be modified by referring to the measured fuel efficiency, which could be negative. | |
| Mar 20, 2022 at 9:46 | comment | added | Igor F. | @JoeBlackSci It is possible and it's being done in practice, but it is a backup option. Domain knowledge is always better. | |
| Mar 20, 2022 at 9:44 | vote | accept | JoeBlackSci | ||
| Mar 20, 2022 at 9:41 | comment | added | JoeBlackSci | So I understand that you should theoretically know the shape of your regression function. I also understand there are probably better ways of determining a model through exploratory analysis than linear regression in that case. But theoretically if one was to use linear regression explorativly as I've done here, could you use a validation dataset to assess the models? Also if you found my question to be clear and well written consider upvoting to allow me to comment and learn more elsewhere. | |
| Mar 20, 2022 at 9:35 | vote | accept | JoeBlackSci | ||
| Mar 20, 2022 at 9:36 | |||||
| Mar 20, 2022 at 9:19 | history | answered | Igor F. | CC BY-SA 4.0 |