# Fit a regression model to the data

A scatter plot is drawn to represent the relationship between response variable y and the predictor variables ‘x’, which is continuous and the indicator variable z which encodes the membership of response to respective group.What should be the regression model equation to fit the data points based on the scatter plot (below) ?

All I can think of is fitting two separate equations for the two populations. Is this approach correct? • Is this a question from a textbook and/or assignment? If so, please add self-study tag and read its wiki. – T.E.G. Nov 7 '16 at 4:26
• Instead of just making a "throwing up your hands" comment include how you've considered the alternatives. You have 2 predictors and one response. What possible models are there? – John Nov 7 '16 at 13:55
• This seems like an unusual dataset for fitting a simple function to, as there are clearly two separate populations. Maybe it would be more appropriate to try and learn the population densities? What is the context in which the data is introduced? – LE Rogerson Nov 7 '16 at 15:23

No, you don't need two equations. You can model that in one equation with z being a dummy variable. From the visual, both groups seem to follow a linear relationship with similar slope but varying intercepts. Or else you might conclude, that both are linear but differ slightly in slope and massively in intercept.

So a bad first approach to model them in one equation as one linear system would look like

$y \leftarrow \beta_0 + \beta_1\times x$

(with "$\leftarrow$" representing some kind of equation with errors left out for simplicity)

Now, far better would be to allow for a different intercept depending on $z$, which is in the following equation

$y \leftarrow \beta_0 + \beta_1\times z +\beta_2\times x$

If $z$ is zero, the intercept will be $\beta_0$, if $z$ is one, it will be $\beta_0+\beta_1$, a different value. If you also want the slope to be different, it's

$y \leftarrow \beta_0 + \beta_1\times z +\beta_2\times x+\beta_3\times x\times z$

You can easily see, that the slope is either $\beta_2$ or $\beta_2+\beta_3$.

So it depends on your assessment of the situation (background knowledge and plot), but through the power of dummy variables, both groups fit into one equation. With all of this being in one equation, you can then fit one modell to all the data and draw conclusions from which terms are significant or not significant.

In R you would modell that as

lm(y ~ x + z) # different intercept
lm(y ~ x + z + x*z)  #different intercept and different slope

• For this solution to work, I believe it is important to mention one more criterion that you seem to have overlooked: the distributions of the errors around each of the separate lines ought to be comparable; in particular, they should have approximately the same standard deviation. Note that this is testable by comparing the variances of the residuals in the two groups. In this particular case it is visually evident from the vertical scatter in both point clouds that this criterion does apply. (This comment pertains to @ira's answer, too.) – whuber Nov 7 '16 at 18:05

This is the case when you should use dummy variable in the regression, because there are two groups following similar trend and only the mean value of the two groups differ.

I use the R programming language in the following example:

# simulate similar data to what you have
x <- rep(c(1:10), 2)
errors <- rnorm(20, 0, 1)
z <- rep(c(-5, 5), 10)
y <- 0.5*x + z + errors

# show a plot
plot(x, y)

fit <- lm(y ~ x + as.factor(z))
# print summary statistics of the model
summary(fit)

# make the whole thing visual
plot(x, y)
# plot the line for group with higher average y
abline(fit$coefficients + fit$coefficients, fit$coefficients, col = "red", lwd = 2) # plot the line for group with lower average y abline(fit$coefficients, fit\$coefficients, col = "blue", lwd = 2)


Here is the summary from the estimated model: And here is the plot to make it visual: • This is really not a good answer, you provide code (which is fine) but the code wont run in any language as is... Further you could dive more into what your method does to solve the problem... just my 2cents – Repmat Nov 7 '16 at 16:29
• You know you can provide actually helpful answer anytime... but you were also right so I have improved my answer a bit. I would gladly improve it further if there are any clarifying questions from the person who raised the original question – ira Nov 7 '16 at 17:02
• Ira, sure I could answer the question... But you already provided an answser, and with your edit it turned out real good. – Repmat Nov 7 '16 at 17:09