4
$\begingroup$

I want to know if there is any regression model for single variable other than simple linear regression. I usually use tree based regression models when there are more than 1 feature and for data with only 1 independent variable, I cant think of any other model other than simple linear model. For simple regression problems, I end up using data transformation into non linear if necessary but I cant think of other distribution free or robust models that work well on single variable model.

My question is, are there any other models (linear or non linear models) that work generally well on single independent variable with 1 dependent variable other than simple linear regression? I tried tree based regressors and MARS on single variable problem but they dont seem to work very well when there are not many features ...

$\endgroup$
1

2 Answers 2

4
$\begingroup$

Well, depending on the nature of the response variable $Y$ there are many possible regression models using only one predictor $x$ (and an intercept, mostly.)

You should read up on generalized linear models, which is a natural starting point. Look at Understanding of GLM, Is there a GLM bible? or search this site.

$\endgroup$
1
$\begingroup$

Using basis expansion one can easily extend simple linear regression into non-linear models.

Here is an example of how basis expansion works (with Fourier and polynomial basis).

Depending on the data, we can chose the right model to fit. In the link, we are trying to fit a periodic data, so it is better to use Fourier basis. Note that it is one independent variable and one dependent variable setting, and it works similar to "multiple regression". But we are fitting a non-linear model.

What's wrong to fit periodic data with polynomials?

$\endgroup$
1
  • $\begingroup$ Thanks for your reply. Lets say outcome variable is continous and I only have 1 independent variable. And, imagine that unless you perform log/boxcox transformation on both dependent and independent variable, the relationship between indepedent and dependent variable is not visible (correlation is only around 0.02). In this scenario, all I can think of is log-log or boxcox-boxcox model. Now, will there be any other model that would generally work in this kind of situation other than linear model? I have read about Fourier but not too sure if that fits on my data. $\endgroup$
    – Tty
    Commented Feb 4, 2020 at 2:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.