What do I do with zero columns in a tensor.prod.model.matrix?

Question

Suppose my two marginal bases are given by (for variable $p$ and $t$ the degree is equal to $1$ and $5$ inner knots are used).

> bp
     p.1  p.2  p.3  p.4  p.5 p.6
[1,]   1 0.00 0.00 0.00 0.00   0
[2,]   0 0.75 0.25 0.00 0.00   0
[3,]   0 0.00 0.50 0.50 0.00   0
[4,]   0 0.00 0.00 0.25 0.75   0
[5,]   0 0.00 0.00 0.00 0.00   1

and

> bt
     t.1  t.2  t.3  t.4  t.5 t.6
[1,]   1 0.00 0.00 0.00 0.00   0
[2,]   0 0.75 0.25 0.00 0.00   0
[3,]   0 0.00 0.50 0.50 0.00   0
[4,]   0 0.00 0.00 0.25 0.75   0
[5,]   0 0.00 0.00 0.00 0.00   1

Obviously defining the tensor-product-matrix as $B_i = bp_i\otimes bt_i$ I will get zero columns if, for instance, the elements of the first column of $bp$, i.e., $p.1$ are multiplied with anything which is not the first column of $bt$, i.e., $t.1$.

The problem here is, that I use some quasi-Newton approach for optimization which uses a predefined gradient which involves multiplications with the columns of the tensor matrix $B$.

Hence if there is a zero column, the gradient will be zero at this point and any abitrary parameter value can be used since the procedure will not change this parameter.

In the optimization step, will those columns just be eliminated?

Answer 1

Note: I wanted to post this as a comment, since it isn't really a complete answer, but it was too long.

Not sure I understand fully. You're only showing five observations here, so it just happens to turn out that, given your choice of marginal bases, some of your tensor product bases only evaluate to zeros (at these five data points). In reality, you must have many more observations, and therefore (if you've chosen your marginal basis dimensions appropriately) every tensor product basis function will have numerous nonzero values. If the full tensor product model matrix actually has columns which have no nonzero values, there is definitely something wrong (perhaps your marginal basis dimension(s) are too high).

As for column elimination, I think most modern regression/modeling software packages will impose identifiability constraints in order to eliminate columns which are linearly dependent on combinations of other columns in the model matrix. But I don't think you'll run into this issue if your marginal dimensions and knot locations are reasonable.