linear regression - polynomial of higher degree with one hot encoding I am trying to fit a hypothesis to my data in order to predict the duration for a certain event.
My data is strictly categorical and I used one-hot encoding for all features.
After using one-hot encoding the dataset now has the dimensions 20.000x414.
After plotting the training and cross validation error as a function of the training set size, I know that my hypothesis suffers from a high bias.
I thought about using a polynomial of higher order but that would not make any difference because all of my features are either '0' or '1'.
Is that assumption correct?
 A: Not too sure what your question is. Could you clarify what are your input features and what you are trying to predict. If your output is binary, I would suggest using softmax function and your objective function for optimization should be a cross-entropy. Using a polynomial regressor is not appropriate in this case.
A: You wrote:    
I thought about using a polynomial of higher order 
but that would not make any difference 
ecause all of my features are either '0' or '1'.

Is that assumption correct?

Your hypothesis (as I understand it):    
If one makes a polynomial fit to data that only contains the values "0" and "1", then a first order polynomial (line) is going to be no more or less effective than an $n^{th}$ order polynomial where $n \ge 2 $ because my values are only 1 and zero.
Question:
Can a polynomial function on the corners of a unit hypercube give better classification accuracy than a linear function?
My approach:
You can think of your data as a existing on the corners of a unit hypercube of dimension equal to the number columns.  
You are wanting to see if you can make a surface through that cube such that points at corners on one side tend to be 1, and on the other side of the surface tend to be 0.
Can I come up with a polynomial surface that will give better dispositioning than a hyperplane?
The curvature gives advantage if there is an idea of interior.  If I can rotate the space, then I could solve the 2d-xor problem with a polynomial, while it has been proved that it requires two separate "perceptrons" (planar cuts of the space).  I can't do it in 2d unless I can make a function like $z \left( x,y \right)$.  
I can do that.  I think that is part of how SVD gets its "powers".  Rotation or parametric expressions are allowed.
Answer:
You can make polynomial functions in the space that solve problems that pure linear ones cannot, in particular the xor problem.  
That doesn't necessarily directly apply to the particulars your particular problem.  I don't know your particular data.  It does speak to the generals of your general question.
A: The assumption is not correct. If you have two features $A$ and $B$, polynomial features of degree 2 will add new features $A^2$, $B^2$, and $AB$ into your dataset. If $A$ and $B$ are binary features, $A^2$, $B^2$ will not make a difference as you said, but $AB$ will make a difference, it will be 1 when both $A$ and $B$ are one. This can help the learning algorithm if there is a relationship between $AB$ and the target.
