This question already has an answer here:
- In linear regression, when is it appropriate to use the log of an independent variable instead of the actual values? 8 answers
- Regression: Transforming Variables 1 answer
- Box-Cox like transformation for independent variables? 3 answers
- When (and why) should you take the log of a distribution (of numbers)? 2 answers
I’m taking an online “Intro to AI” course for which I’m doing some azure machine learning labs. This course is largely about how to apply azure ML solutions and, while there is an “essential math for ML module”, it doesn’t do any sort of deep dive into the math/statistics. But I want to understand what’s happening mathematically to a basic level. (I've taken an intro to statistics, that's it so far.) Below are some details about some specific things I’d like to understand more, and my specific questions.
At a particular point in the “training a classification model” lab, there is this text: “The distribution of the Age column in the diabetes.csv dataset is skewed because most patients are in the youngest age bracket. Creating a version of this feature that uses a natural log transformation can help create a more linear relationship between Age and other features, and improve the ability to predict the Diabetic label. This kind of feature engineering as it’s called is common in machine learning data preparation.”
Question #1: What is that transformation actually doing? I don't mean the nitty gritty math, but what is it doing conceptually?
Question #2: The next general question is around why is a transformation necessary. On that, I did some research and found this article (https://www.r-statistics.com/2013/05/log-transformations-for-skewed-and-wide-distributions-from-practical-data-science-with-r/) that describes which type of log transformations to use in a few scenarios. Here’s a snip of text from the article: “The need for data transformation can depend on the modeling method that you plan to use. For linear and logistic regression, for example, you ideally want to make sure that the relationship between input variables and output variables is approximately linear, that the input variables are approximately normal in distribution, and that the output variable is constant variance (that is, the variance of the output variable is independent of the input variables). You may need to transform some of your input variables to better meet these assumptions.”
I don’t understand why any of that is necessary. I’ve broken my question down:
For linear and logistic regression, for example, you ideally want to make sure that: the relationship between input variables and output variables is approximately linear – why? the input variables are approximately normal in distribution- why? the output variable is constant variance (that is, the variance of the output variable is independent of the input variables – why?
Could someone help with information for those whys, or point me to material that would help?