Non-parametric regression models and data transformations

Question

I am trying to explain datasets gathered from simulation with various regression models. Linear regression seems not applicable because my data after transformations still violate the assumptions for linearity, normality of errors etc.

I have built several non-parametric models (splines, regression trees, SVR and Gaussian processes), however I am not quite sure if I should transform the data or not for the analysis. From the theory I have so far understood that non-parametric models make no / very few assumptions about the structure of underlying data, however it is not clear to me if this means that I can apply non-parametrics to my original data without transformations of any kind. What does the term "few" assumptions mean? Unfortunately I haven't managed to find something more concrete on this.

Another question is that splines for example derive the model with the least squares method. How is it possible that in this case no assumptions on the data are necessary as opposed to linear regression?

Answer 1

To answer your second question, the difference is that with linear regression you are assuming the response is a linear function of the predictors. With a spline you make no such assumption about the functional form. However, you still need to make various assumptions in the construction of the spline such as continuity at the knot points and, as you noted, the cost function.