I have a few related questions that have been bugging me for quite a while regarding non-linearity in linear & logistic regression with multiple predictors.

EDIT: I have since removed parts 3 and 4 of the question (will post separately).

1. Visualizing non-linearity in multiple linear/logistic regression

When building regression/classification predictive models with multiple predictors, one of the things I have never fully understood is if one can visually determine when a transformation is appropriate on the predictors.

It is clear when plotting $y \times x$ for simple linear regression where a relationship could be non-linear and a log/square-root/polynomial/spline transformation of $x$ can help model this non-linearity, but does this logic extend reliably to multiple regression? Could the observed non-linearity not be explained away by other predictors in the model?

Every text I read seems to only talk about non-linear transformations in the simple linear/logistic regression scenario, so it's not clear to me whether I can just extend this logic in the presence of other predictors and still expect model improvement. I guess an equivalent question but reversed would be "if a linear fit is best in the simple linear regression case, will it also be best in the presence of other predictors for multiple regression?"

For example, if I am building a multiple regression

$$y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$$

If I plot the relationship between $y$ and $x_3$ and think "hmm, this relationship is non-linear, perhaps I should add a second/third-order term for $x_3$ or use a spline basis with 4 knots", is it reasonable to assume this will also be a good transformation in multiple regression? Even if there are cases where this isn't the case, would you say it is still a reasonable strategy, or totally pointless?

2. Visualizing non-linearity (logistic, specifically)

Furthermore, if the above approach is reasonable, is there a similarly reliable way to visually determine non-linearity with the logit? I tried an approach for assessing linearity in logistic regression (could be misinformed) which involves binning numeric predictors before into equal-spaced bins, e.g. if we are fitting

$$ln \left(\frac{p}{1-p} \right) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$$

I thought I could perhaps bin $x_1$ into equal-range bins, say $[0, 5), [5, 10), \dots, [25, 30)$, calculate $p$ as the proportion of events across each bin, then the logodds $ln \left(\frac{p}{1-p} \right)$. I would then plot the log odds across bins to assess if linearity is reasonable, but the arbitrary selection of how wide the bins are changes how non-linear the relationship looks which usually puts me off this approach.

I have a few related questions that have been bugging me for quite a while regarding non-linearity in linear & logistic regression with multiple predictors.

EDIT: I have since removed parts 3 and 4 of the question (will post separately).

1. Visualizing non-linearity in multiple linear/logistic regression

When building regression/classification predictive models with multiple predictors, one of the things I have never fully understood is if one can visually determine when a transformation is appropriate on the predictors.

It is clear when plotting $y \times x$ for simple linear regression where a relationship could be non-linear and a log/square-root/polynomial/spline transformation of $x$ can help model this non-linearity, but does this logic extend reliably to multiple regression? Could the observed non-linearity not be explained away by other predictors in the model?

Every text I read seems to only talk about non-linear transformations in the simple linear/logistic regression scenario, so it's not clear to me whether I can just extend this logic in the presence of other predictors and still expect model improvement. I guess an equivalent question but reversed would be "if a linear fit is best in the simple linear regression case, will it also be best in the presence of other predictors for multiple regression?"

For example, if I am building a multiple regression

$$y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$$

If I plot the relationship between $y$ and $x_3$ and think "hmm, this relationship is non-linear, perhaps I should add a second/third-order term for $x_3$ or use a spline basis with 4 knots", is it reasonable to assume this will also be a good transformation in multiple regression? Even if there are cases where this isn't the case, would you say it is still a reasonable strategy, or totally pointless?

2. Visualizing non-linearity (logistic, specifically)

Furthermore, if the above approach is reasonable, is there a similarly reliable way to visually determine non-linearity with the logit? I tried an approach for assessing linearity in logistic regression (could be misinformed) which involves binning numeric predictors before into equal-spaced bins, e.g. if we are fitting

$$ln \left(\frac{p}{1-p} \right) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3$$

I thought I could perhaps bin $x_1$ into equal-range bins, say $[0, 5), [5, 10), \dots, [25, 30)$, calculate $p$ as the proportion of each bin that is an 'event', then the log odds $ln \left(\frac{p}{1-p} \right)$. I would then plot the log odds across the bins to assess if linearity is reasonable.

The problem is I perceive this approach having the same issues as in the regression case in part 1. (if they are indeed issues), and the arbitrary selection of how wide the bins are changes how non-linear the relationship looks. Both of these together usually puts me off using this approach at all.

I have a few related questions that have been bugging me for quite a while regarding non-linearity in linear & logistic regression with multiple predictors. I apologize for the length, but I didn't think some of these justified being their own questions and some might only require very brief answers. I have highlighted the key points in bold.

Furthermore, if the above approach is reasonable, is there a similarly reliable way to visually determine non-linearity with the logit? I tried an approach for binary classification (could be misinformed) which involves binning numeric predictors before into equal-spaced bins, e.g. if we are fitting

I thought I could perhaps bin $x_1$ into equal-range bins, say $[0, 5), [5, 10), \dots, [25, 30)$, calculate $p$ as the proportion of events across each bin, then the logodds $ln \left(\frac{p}{1-p} \right)$. I would then plot the log odds across bins to assess if linearity is reasonable, but the arbitrary selection of how wide the bins are changes how non-linear the relationship looks which usually puts me off this approach.

3. If you could test just one transformation, would it be a cubic/natural cubic spline?

I've found myself wondering "if i'm interested in predictive power and suspect non-linearity between a certain predictor and the response, is it reasonable to just default to testing a cubic/natural cubic spline transformation with a few knots?" From what I've seen so far these seem to model non-linearity more reliably with sufficient data than log/polynomial/square-root transformations, still don't add too many degrees of freedom and don't require me to test a wide range of transformations for each predictor.

4. Cross-validation-based approach to finding non-linear transformations?

Because I have never been sure if the visualization approach works well for multiple linear & logistic regression, I tend to take my multiple linear/logistic model with selected variables and interactions and try replacing each of these linear terms one-by-one with the basis for a natural cubic spline for that variable with a few knots (usually 3 at the 25th, 50th and 75th percentiles).

I then compare the repeated cross-validation performance to the original regression (with no spline terms) using the same folds. I then rank-order by which variables improved the cross-validation metric (say ROC AUC, RMSE) the most when using a spline transformation. I use this as a basis for deciding which terms (if any) would benefit from non-linear transformations in the presence of the variables already in my model. Does this sound like a reasonable approach?

I have a few related questions that have been bugging me for quite a while regarding non-linearity in linear & logistic regression with multiple predictors.

EDIT: I have since removed parts 3 and 4 of the question (will post separately).

Furthermore, if the above approach is reasonable, is there a similarly reliable way to visually determine non-linearity with the logit? I tried an approach for assessing linearity in logistic regression (could be misinformed) which involves binning numeric predictors before into equal-spaced bins, e.g. if we are fitting

I thought I could perhaps bin $x_1$ into equal-range bins, say $[0, 5), [5, 10), \dots, [25, 30)$, calculate $p$ as the proportion of events across each bin, then the logodds $ln \left(\frac{p}{1-p} \right)$. I would then plot the log odds across bins to assess if linearity is reasonable, but the arbitrary selection of how wide the bins are changes how non-linear the relationship looks which usually puts me off this approach.