2
$\begingroup$

In this scenario, one group has a clear linear relationship between x and y. Another group doesn't. However, there is no way to differentiate them in the data. In this case, performing a simple linear regression on the data results in a clear trend in the residual plot. A simple linear regression isn't appropriate in this case and transformations of the variables is meaningless right?

Here's my R code to simulate this scenario, in order to show what I'm talking about:

    num_obs = 100
    beta_0  = 0
    beta_1  = 1
    sigma   = 2
    set.seed(1)
    epsilon = rnorm(n = num_obs, mean = 0, sd = sigma)
    x_vals = seq(from = 0, to = 10, length.out = num_obs)#group 1
                 # (linear relationship between x and y)
    y_vals = beta_0 + beta_1 * x_vals + epsilon # group 1
    
    x_vals = c(x_vals,rep(c(0:10),10))   #add group 2  (no 
             # relationship between x and y)
    y_vals = c(y_vals, rep(0:9, each=11))  #add group 2
    
    sim_fit = lm(y_vals ~ x_vals)
    coef(sim_fit)
    plot(y_vals ~ x_vals)
    abline(sim_fit)
    
    plot(sim_fit$fitted.values, sim_fit$residuals) #residual plot
                         # shows clear trend

Here's the data and the best fit line:

enter image description here

$\endgroup$
1
  • 1
    $\begingroup$ Could you tell us context and details? You say you have two groups, but dont know which datapoints pertains to which group? Is that correct? $\endgroup$ Commented Aug 7, 2021 at 0:40

1 Answer 1

3
$\begingroup$

You seem to have two groups, with different regressions within each group, but then you have lost the group id's. If that is the situation, one approach is to do clustering and regression simultaneously. One way to do that is via an EM algorithm (Expectation Maximization). There are at least two R packages implementing this, mixreg and flexmix, and I will make a small illustration using the first one, and the data you simulated.

So, starting after running the code in your post,

  library(mixreg)
  df <- data.frame(x=x_vals, y=y_vals)
  mixmod <- mixreg::mixreg(df$x, df$y, ncomp=2)

A plot produced by the package is

mixreg plot of two lines with conf bands

which clearly shows the two distinct regression lines. References for the methods can be found in the package documentation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.