# Algorithms to model non-linear relationship between two vectors

I want to build a model that describes a curve that fits the data shown in the scatterplot. I thought it would be straight forward using sklearn. But the choice and application of the different methods gets rather confusing.

Which algorithms would you use to tackle this problem?

• Welcome to our site. Could you explain the sense in which you use "vectors" in your title? This appears to be a (standard) univariate regression problem, not a vector problem. – whuber Sep 4 '18 at 13:40
• Vector in a computational sense. c(1,2,3,4,5) or [1,2,3,4,5] or array(1,2,3,4,5) – Benni Sep 4 '18 at 19:58
• The only way that would apply here would be to represent the $x$ coordinates as one vector and the $y$ coordinates as another. But what, then, could you possibly mean by a "nonlinear" relationship between the two of them? – whuber Sep 4 '18 at 20:35

I suppose the answer depends on what you are trying to accomplish:

• Predicting future values
• Performing statistical inference
• Other ?

My answer is that you can use any of the many regression models available and choose the one that you believe to be the most appropriate using whichever metric you are comfortable with. Here are a few examples along with the Python Sklearn code

### Polynomial linear regression

import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline, make_pipeline

model_1 = make_pipeline(PolynomialFeatures(degree = 5),LinearRegression())
model_1.fit(x.reshape(-1,1),y)
plt.figure(figsize = (8,5))
plt.scatter(x,y, alpha = .3, label = 'Data')
plt.plot(x,model_1.predict(x.reshape(-1,1)), color = 'red', label = 'Model')
plt.title('Polynomial degree 5')
plt.xlabel('X'), plt.ylabel('Y')
plt.legend(), plt.show()


## Decision tree regression

from sklearn.tree import DecisionTreeRegressor
model_2 = DecisionTreeRegressor(max_depth = 3)
model_2.fit(x.reshape(-1,1),y)


## Piecewise linear spline interpolation

from scipy import interpolate
tck = interpolate.splrep(x, y, k=1, s=1, t = [35])
plt.plot(x,interpolate.splev(x, tck, der=0), color = 'red', label = 'Model')


• Thank you for your clear answer. I'm certainly searching for a smooth curve in my model to predict y from all possible x. The polynomial regression is the first step. – Benni Sep 4 '18 at 20:00
• You can also try LOESS regression and cubic splines if you are looking for smooth models - wont be able to do as much inference on those though – Xavier Bourret Sicotte Sep 4 '18 at 21:51