# linear regression - polynomial of higher degree with one hot encoding

I am trying to fit a hypothesis to my data in order to predict the duration for a certain event.

My data is strictly categorical and I used one-hot encoding for all features.

After using one-hot encoding the dataset now has the dimensions 20.000x414.

After plotting the training and cross validation error as a function of the training set size, I know that my hypothesis suffers from a high bias.

I thought about using a polynomial of higher order but that would not make any difference because all of my features are either '0' or '1'.

Is that assumption correct?

You wrote:

I thought about using a polynomial of higher order
but that would not make any difference
ecause all of my features are either '0' or '1'.

Is that assumption correct?


Your hypothesis (as I understand it):

If one makes a polynomial fit to data that only contains the values "0" and "1", then a first order polynomial (line) is going to be no more or less effective than an $n^{th}$ order polynomial where $n \ge 2$ because my values are only 1 and zero.

Question:
Can a polynomial function on the corners of a unit hypercube give better classification accuracy than a linear function?

My approach:
You can think of your data as a existing on the corners of a unit hypercube of dimension equal to the number columns.

You are wanting to see if you can make a surface through that cube such that points at corners on one side tend to be 1, and on the other side of the surface tend to be 0.

Can I come up with a polynomial surface that will give better dispositioning than a hyperplane?

The curvature gives advantage if there is an idea of interior. If I can rotate the space, then I could solve the 2d-xor problem with a polynomial, while it has been proved that it requires two separate "perceptrons" (planar cuts of the space). I can't do it in 2d unless I can make a function like $z \left( x,y \right)$.

I can do that. I think that is part of how SVD gets its "powers". Rotation or parametric expressions are allowed.

The assumption is not correct. If you have two features $$A$$ and $$B$$, polynomial features of degree 2 will add new features $$A^2$$, $$B^2$$, and $$AB$$ into your dataset. If $$A$$ and $$B$$ are binary features, $$A^2$$, $$B^2$$ will not make a difference as you said, but $$AB$$ will make a difference, it will be 1 when both $$A$$ and $$B$$ are one. This can help the learning algorithm if there is a relationship between $$AB$$ and the target.