2
$\begingroup$

Polynomial regression is said to be a subclass of linear regression. However, it seems that only linear regression can handle data input with more than one dimension, whereas polynomial regression cannot. Is my understanding correct?

For example,

Given a set of $N$ data pairs $\{(x_k,t_k)\}_{k = 1}^N, x_k \in \mathbb{R}^n, t_k\in \mathbb{R}$

A linear regression uses the model

$y(x,t) = \sum\limits_{i = 1}^n w_i x_i$ where $w_i$ are the coefficients of my model

Then the empirical loss is written as

$E = \|Xw - t\|$^2

where $X = \begin{bmatrix} 1 & x_1^T \\ \vdots & \vdots \\ 1 & x_N^T \end{bmatrix}$, $w = \begin{bmatrix} 1 & w_1 & \ldots & w_n\end{bmatrix}^T$

In the Polynomial regression case, my model is,

$y(x,t) =\sum\limits_{i = 0}^n w_ix^i = w_0 + w_1x + w_2x^2 + \ldots w_nx^n$

Here, $x$ no longer represents the entries of a vector $x \in \mathbb{R}^n$. But just a single point $x \in \mathbb{R}$.

I think the issue here is that powers are not defined for non-1D vectors.

$\endgroup$
  • $\begingroup$ Powers are defined: they are called "tensor products." $\endgroup$ – whuber Jan 25 at 13:32
0
$\begingroup$

Here is an example 3D surface quadratic polynomial and data. surface

scatter

$\endgroup$
  • $\begingroup$ How would you formulate this problem in terms of the loss? $\endgroup$ – Carlos - the Mongoose - Danger Jan 25 at 14:28
  • $\begingroup$ instead of "y=f(x)", I used "z=f(x,y)" and standard lowest sum-of-squared-error linear regression. For example, a flat surface would be "z = a + bx + cy". $\endgroup$ – James Phillips Jan 25 at 16:27
0
$\begingroup$

I will quote wikipedia because it is really well said.

enter image description here

you can find this article here : https://en.wikipedia.org/wiki/Polynomial_regression

You will see here how you obtain it : http://mathworld.wolfram.com/LeastSquaresFittingPolynomial.html

and here on a more mathematic point of view: http://home.iitk.ac.in/~shalab/regression/Chapter12-Regression-PolynomialRegression.pdf

$\endgroup$
  • $\begingroup$ But Wikipedia doesn't contradict what I said. Look, each row of the matrix $X$ looks like $[1, x_1, x_1^2, \ldots, x_1^m]$. But in linear regression, each row of the matrix $X$ consists of all the elements of the vector $[1, x_{11}, x_{12}, \ldots, x_{1m}]$. These are just totally different objects $\endgroup$ – Carlos - the Mongoose - Danger Jan 25 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.