I do not have a specific example or task in mind. I'm just new on using b-splines and I wanted to get a better understanding of this function in the regression context.
Let's assume that we want to assess the relationship between the response variable $y$ and some predictors $x_1, x_2,...,x_p$. The predictors include some numerical variables as well as some categorical ones.
Let's say that after fitting a regression model, one of the numerical variables e.g $x_1$ is significant. A logical step afterwards is to assess whether higher order polynomials e.g: $x_1^2$ and $x_1^3$ are required in order to adequately explain the relationship without overfitting.
My questions are:
At what point do you chose between b-splines or simple higher order polynomial. e.g in R:
y ~ poly(x1,3) + x2 + x3
y ~ bs(x1,3) + x2 + x3
How can you use plots to inform your choice between those two and what happens if it's not really clear from the plots (e.g: due to massive amounts of data points)
How would you assess the two-way interaction terms between $x_2$ and let's say $x_3$
How do the above change for different types of models
Would you consider to never use high order polynomials and always fitting b-splines and penalise the high flexibility?