2
$\begingroup$

I checked for related questions before posting this. I found some similar questions (like this), but I couldn't find any question that answers mine.

I have two independent variables, one continuous and one categorical (1 or 0). My dependent variable is continuous in [0,1].

I want to analyze the effect of each independent variable. I understood I should apply regression analysis. However, according to resources I read like here and here, there is more than one way to do that. I tried to do Linear Regression (Ordinary Least Squares). The $R^2$ I obtained is 0.509. Here is the residual plot I obtained:

enter image description here

I think that $R^2$ is not good enough. So, I want to improve my model. I tried to analyze the residual plot by using information from here. Unfortunately, I think my plot doesn't look like any other plot provided on the last link I shared. It is said that there shouldn't be any pattern in the residual plot but I cannot decide whether my plot contains some kind of pattern.

To sum up, I don't know what should I do to improve my model: Should I try to use a non-linear model? Should I transform my data points ?

EDIT

As requested in the comments I put some other figures:

1) X0 - Y chart

enter image description here

2) X1 - Y chart

enter image description here

3) Some of the data points I have

enter image description here

$\endgroup$
  • $\begingroup$ @NickCox Hi again, sorry I was late to edit my question. I put plots of Y vs X1 and Y vs X0. I also shared some of the data points I have. Unlike in the original version of my post, I use more data points now (I forgot to use some of them initially). In this version I have around 400 data points. Unfortunatelly my R2 value descreased to 0.37 now. Do you have any suggestion ? $\endgroup$ – zwlayer Dec 10 '18 at 20:41
1
$\begingroup$

A regression model $Y = a + b_0 X_0 + b_1 X_1$ (the slightly awkward and non-standard notation matches your predictor names) plots as $a + b_0 X_0$ for $X_1 = 0$ and $a + b_0 X_0 + b_1$ for $X_1 = 1$, i.e. two parallel lines in $X_0, Y$ space. Here you can see a tilt in the main cluster of points which does not match the regression especially well, but you can also see that data points in the NE and E corners of each plot exert leverage pulling up the regression surface.

enter image description here

Although not shown here, a quadratic in $X_0$ (with $X_1$ as extra predictor in the same way) does not seem to offer worthwhile improvement. I don't have other models to suggest, nor do I suggest any transformations. It's absolute that $X_1$ cannot be transformed usefully (as any pair of values other than 0 and 1 would yield equivalent models), while it's hard to envisage a useful transformation for $Y$ given its narrow range. $X_0$ is the most obvious focus of attention.

The narrow range of the outcome from about 0.48 to 0.55 seems unusual, at least until the OP offers some kind of explanation. We have no context here on what kinds of relationship are substantively possible or plausible (or conversely impossible or implausible).

For $R^2$ Stata gives me 0.3725.

$\endgroup$
2
$\begingroup$

I think a lot of people look at R-squared as a goal post (the higher the better). In my opinion, this is a little misguided. The R-squared is intended to tell you what proportion of variance your covariates explain. It can be shoe horned into a model fit metric, but it isn't a good idea. R squared can be low simply because the data are noisy, not because your model is a bad fit. I think that is what is happening here.

Take a look at your residual plot. There are a lot of points around 0.52, but they take on wildly different values. No amount of non-linear modelling will help reduce that variance without overfitting the hell out of the data.

If you are insistent about using non-linear methods, some black box ML method might to best. I can't say more without having the data myself, however.

$\endgroup$
  • $\begingroup$ 'overfitting the hell' (not a standard term of art, but we get the sense) isn't being suggested by anyone. I wouldn't rule out another parameter e.g. a quadratic, if it were consistent with scientific or other context on behaviour. $\endgroup$ – Nick Cox Dec 10 '18 at 21:13
  • $\begingroup$ @DemetriPananos how can I be sure that my model fits well to the dataset then ? $\endgroup$ – zwlayer Dec 10 '18 at 23:02
  • $\begingroup$ @zwlayer You have to assess the assumptions of the regression. If all the assumptions seem to be reasonably satisfied, then you can say the model fits well. Wether or not the model predicts the data well is a different sort of question. $\endgroup$ – Demetri Pananos Dec 10 '18 at 23:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.