# Building a regression model for non-linear relationship

I'm trying to build a regression model that best explains the variable C by a function of A and B. Below is the descriptive statistics of the dataset. There are total of 300 instances in this data.

  A                 B                 C
Min.   :-8.8600   Min.   :-2.8900   Min.   :-10000.00
1st Qu.:-1.9400   1st Qu.:-0.6062   1st Qu.:   -23.38
Median : 0.4685   Median : 1.9350   Median :    -3.09
Mean   : 2.3170   Mean   : 2.0131   Mean   :   -41.56
3rd Qu.: 7.0425   3rd Qu.: 4.6975   3rd Qu.:    16.30
Max.   :20.6000   Max.   : 7.0000   Max.   :   202.00


From the summary descriptives, and from boxplots and scatterplots, I identified an outlier in the column C. I proceeded to remove the outlier and got the below summary.

       A                B                C
Min.   :-8.860   Min.   :-2.890   Min.   :-315.000
1st Qu.:-1.950   1st Qu.:-0.587   1st Qu.: -23.100
Median : 0.447   Median : 1.940   Median :  -2.940
Mean   : 2.291   Mean   : 2.027   Mean   :  -8.259
3rd Qu.: 7.025   3rd Qu.: 4.705   3rd Qu.:  16.300
Max.   :20.600   Max.   : 7.000   Max.   : 202.000


Below are the scatterplots between A and C, and B and C from the dataset without the outlier.

You can see from the plot that B has somewhat of a linear relationship with C, but A does not. I tried fitting polynomial models with A^2 and A^3, but the adjusted R-square does not increase noticeably, with it only being somewhere around 0.4.

In such case, what approaches might I take? I've attached the result of some of the basic regression models that I have fitted below for reference.

Call:
lm(formula = C ~ A + B, data = dat2)

Residuals:
Min      1Q  Median      3Q     Max
-229.41  -26.36    1.93   32.99  166.43

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  25.7320     4.2640   6.035 4.75e-09 ***
A            -1.3667     0.5776  -2.366   0.0186 *
B           -15.2219     1.0974 -13.871  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 55.71 on 296 degrees of freedom
Multiple R-squared:  0.394, Adjusted R-squared:  0.3899
F-statistic: 96.23 on 2 and 296 DF,  p-value: < 2.2e-16

Call:
lm(formula = C ~ A2 + A + B, data = dat2)

Residuals:
Min      1Q  Median      3Q     Max
-221.44  -22.79   -2.73   33.87  187.43

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  32.31815    4.69006   6.891 3.36e-11 ***
A2           -0.30079    0.09525  -3.158  0.00175 **
A             0.96274    0.93170   1.033  0.30230
B           -15.54204    1.08585 -14.313  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 54.88 on 295 degrees of freedom
Multiple R-squared:  0.4138,    Adjusted R-squared:  0.4079
F-statistic: 69.42 on 3 and 295 DF,  p-value: < 2.2e-16

Call:
lm(formula = C ~ A3 + A2 + A + B, data = dat2)

Residuals:
Min       1Q   Median       3Q      Max
-216.089  -21.384   -2.757   32.428  182.474

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  36.97146    5.25297   7.038 1.38e-11 ***
A3            0.02190    0.01133   1.932  0.05428 .
A2           -0.59956    0.18137  -3.306  0.00106 **
A             0.75143    0.93383   0.805  0.42166
B           -15.73565    1.08549 -14.496  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 54.63 on 294 degrees of freedom
Multiple R-squared:  0.4212,    Adjusted R-squared:  0.4133
F-statistic: 53.48 on 4 and 294 DF,  p-value: < 2.2e-16
$$$$
`
• Where does “multivariate” come up? “Multivariate” regression means a multivariate response variable, but many people say it when they mean multiple predictors. However, your regression does not seem to be either.
– Dave
Apr 27, 2020 at 23:11
• @Dave I edited the title. Mine has 2 predictors which is why I used the word "multivariate". Thanks for pointing that out. Apr 28, 2020 at 0:41

From the two plots you've provided we see that, marginally, as the absolute value of $$A$$ or $$B$$ increases, the variance of $$C$$ increases dramatically, but the expected value of $$C$$ does not change much, linearly with the regressors or otherwise.
As a result, if there is no interaction between the effects of $$A$$ and $$B$$, you will have to accept that regressors $$A$$ and $$B$$ do not explain the mean of $$C$$ very well.
However, it is possible that the interaction of $$A$$ and $$B$$ could have some explanatory value. Perhaps when $$B < 0$$ and $$A > 0$$, the mean of $$Y$$ is large whereas when $$B > 0$$ and $$A > 0$$, the mean of $$Y$$ is small.
To test this, use $$A*B$$ insead of $$A + B$$ in your R formula.