# Linear Regression with dependent variables [closed]

I want to create a linear regression model using two variables, var $$a$$ and var $$b$$, and the coefficients are $$w$$ and $$(1-w)$$ respectively.

So the output dependent variable $$Y = wa + (1-w)b$$.

I am not sure how to approach this. Please suggest.

• What in particular are you not sure of? Commented Mar 7, 2019 at 17:45

The approach is quite simple: algebra. Just to be clear: $$w$$ meaning weight is the coefficient term to be estimated from regression. Correct?

The linear expression reduces to $$Y = w (a-b) + b$$. I am assuming this all has a normal error term or some other rationale for using regression. Anyway, the constant term "b" can be handled with an offset term. And you must create a new variable "c" as "a-b" so that the linear model just adjusts for c and offset(b) and possibly no intercept term.

@AdamO has the right idea. Your setup can be dramatically simplified with some algebra.

$$Y = wA + (1-w)B$$

Implies that

$$Y = w(A-B) + B$$

Which in turn implies

$$Y-B = w(A-B)$$

Assuming you have some errors in there (and maybe a constant?) you should have:

$$Y-B = c + w(A-B) + e$$

This regression can be run very simply. In R:

y_less_b = y-b
a_less_b = a-b
mod = lm(y_less_b~a_less_b)
mod_no_constant = lm(y_less_b~a_less_b-1)


And all the summary stats apply. If you wanted $$w \in [0,1]$$, and that regression doesn't put it there, then a) rethink whether it needs to be in $$[0,1]$$, and/or b) project it to the nearest point in that set (either 0 or 1). Broadly speaking, if it belongs in $$[0,1]$$, its extremely likely to be estimated in that region once you have even a mediocre sample size -- unless your error variance is off the charts.

Here is a Python 3D fitter using your equation with some test data. This example has a 3D scatter plot, a 3D surface plot, and a contour plot.

import numpy, scipy, scipy.optimize
import matplotlib
from mpl_toolkits.mplot3d import  Axes3D
from matplotlib import cm # to colormap 3D surfaces from blue to red
import matplotlib.pyplot as plt

graphWidth = 800 # units are pixels
graphHeight = 600 # units are pixels

# 3D contour plot lines
numberOfContourLines = 16

def SurfacePlot(func, data, fittedParameters):
f = plt.figure(figsize=(graphWidth/100.0, graphHeight/100.0), dpi=100)

matplotlib.pyplot.grid(True)
axes = Axes3D(f)

x_data = data[0]
y_data = data[1]
z_data = data[2]

xModel = numpy.linspace(min(x_data), max(x_data), 20)
yModel = numpy.linspace(min(y_data), max(y_data), 20)
X, Y = numpy.meshgrid(xModel, yModel)

Z = func(numpy.array([X, Y]), *fittedParameters)

axes.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap=cm.coolwarm, linewidth=1, antialiased=True)

axes.scatter(x_data, y_data, z_data) # show data along with plotted surface

axes.set_title('Surface Plot (click-drag with mouse)') # add a title for surface plot
axes.set_xlabel('X Data') # X axis data label
axes.set_ylabel('Y Data') # Y axis data label
axes.set_zlabel('Z Data') # Z axis data label

plt.show()
plt.close('all') # clean up after using pyplot or else thaere can be memory and process problems

def ContourPlot(func, data, fittedParameters):
f = plt.figure(figsize=(graphWidth/100.0, graphHeight/100.0), dpi=100)

x_data = data[0]
y_data = data[1]
z_data = data[2]

xModel = numpy.linspace(min(x_data), max(x_data), 20)
yModel = numpy.linspace(min(y_data), max(y_data), 20)
X, Y = numpy.meshgrid(xModel, yModel)

Z = func(numpy.array([X, Y]), *fittedParameters)

axes.plot(x_data, y_data, 'o')

axes.set_title('Contour Plot') # add a title for contour plot
axes.set_xlabel('X Data') # X axis data label
axes.set_ylabel('Y Data') # Y axis data label

CS = matplotlib.pyplot.contour(X, Y, Z, numberOfContourLines, colors='k')
matplotlib.pyplot.clabel(CS, inline=1, fontsize=10) # labels for contours

plt.show()
plt.close('all') # clean up after using pyplot or else thaere can be memory and process problems

def ScatterPlot(data):
f = plt.figure(figsize=(graphWidth/100.0, graphHeight/100.0), dpi=100)

matplotlib.pyplot.grid(True)
axes = Axes3D(f)
x_data = data[0]
y_data = data[1]
z_data = data[2]

axes.scatter(x_data, y_data, z_data)

axes.set_title('Scatter Plot (click-drag with mouse)')
axes.set_xlabel('X Data')
axes.set_ylabel('Y Data')
axes.set_zlabel('Z Data')

plt.show()
plt.close('all') # clean up after using pyplot or else thaere can be memory and process problems

def func(data, w):
a = data[0]
b = data[1]
return w*a + (1-w)*b

if __name__ == "__main__":
xData = numpy.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0])
yData = numpy.array([11.0, 12.1, 13.0, 14.1, 15.0, 16.1, 17.0, 18.1, 90.0])
zData = numpy.array([1.1, 2.2, 3.3, 4.4, 5.5, 6.6, 7.7, 8.0, 9.9])

data = [xData, yData, zData]

initialParameters = [1.0] # same as scipy default values in this example

# here a non-linear surface fit is made with scipy's curve_fit()
fittedParameters, pcov = scipy.optimize.curve_fit(func, [xData, yData], zData, p0 = initialParameters)

ScatterPlot(data)
SurfacePlot(func, data, fittedParameters)
ContourPlot(func, data, fittedParameters)

print('fitted prameters', fittedParameters)

modelPredictions = func(data, *fittedParameters)

absError = modelPredictions - zData

SE = numpy.square(absError) # squared errors
MSE = numpy.mean(SE) # mean squared errors
RMSE = numpy.sqrt(MSE) # Root Mean Squared Error, RMSE
Rsquared = 1.0 - (numpy.var(absError) / numpy.var(zData))
print('RMSE:', RMSE)
print('R-squared:', Rsquared)