1
$\begingroup$

I have encountered a problem that the literature suggests linear regression is able to solve, but I am at a loss.

I have a function $F$ that I want to estimate. This function obeys $N$ equations of the form $$y_i = \int_0^1 F(x) K_i(x) \; \text{d}x \qquad i=1\ldots N$$ where $y_i$ are known data values and $K_i(x)$ are known functions.

I can represent $F$ using cubic B-splines:

$$ F(x) = \sum_{i=-1}^{M+1} a_i B_i (x). $$

I want to approximate $F$ by estimating those coefficients $a$. Hence the first equation can be rewritten as $N$ equations of the following form $$ y_i = a_1 \int_0^1 B_1(x) K_i(x) \;\text{d}x + \ldots + a_M \int_0^1 B_M(x) K_i(x) \;\text{d}x \qquad i=1\ldots N. $$ Now, as I said, the literature says this problem can be solved via linear regression. This would seem to be the case because the coefficients we are estimating are linear in the target. However, what confuses me is that each cubic spline $B$ has to use those coefficients $a$ in order to calculate the result. So how is this linear? What am I doing wrong here?

$\endgroup$
  • $\begingroup$ I don't see how linear regression would solve this, but after substituting the spline basis you have a straightforward linear algebra problem. $\endgroup$ – jwimberley Jan 23 '17 at 13:34
  • $\begingroup$ @jwimberley I guess I just mean least squares - sorry to cross those wires. However I am still lost. The spline basis is made up of those coefficients $a$ and so the $a$s will appear inside of the integral, won't they? $\endgroup$ – rhombidodecahedron Jan 23 '17 at 16:04
  • 1
    $\begingroup$ Not if the spline basis is linear, as you seem to have indicated here. In your own equations, a factors out of the integral(s). $\endgroup$ – jwimberley Jan 23 '17 at 16:05
1
$\begingroup$

However, what confuses me is that each cubic spline $B$ has to use those coefficients $a$ in order to calculate the result. So how is this linear?

This is not true. As you have stated the problem, only $F$ requires knowledge of $a$ for its computation. Each $B_i$ is a cubic spline with no free parameters. Now, this isn't an appropriate problem for linear regression, but it is a straightforward linear algebra problem. Your main equation can be rewritten in matrix form: $$ \left( \begin{array}{ccc} \int_0^1 B_1(x) K_1(x) \, dx & \int_0^1 B_2(x) K_1(x) & \cdots & \int_0^1 B_M(x) K_1(x) \\ \int_0^1 B_1(x) K_2(x) \, dx & \int_0^1 B_2(x) K_2(x) & \cdots & \int_0^1 B_M(x) K_2(x) \\ \vdots & \vdots & \ddots & \vdots \\ \int_0^1 B_1(x) K_N(x) \, dx & \int_0^1 B_2(x) K_N(x) & \cdots & \int_0^1 B_M(x) K_N(x) \\ \end{array} \right) \left( \begin{array}{c} a_1 \\ y_2 \\ \vdots \\ a_M \end{array} \right) = \left( \begin{array}{c} y_1 \\ y_2 \\ \vdots \\ y_N \end{array} \right) $$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks. I found the problem in my thinking of the concept, which is that the resource I was using gave an example of B_i of order 3 and plugged in all of the coefficients, so it used 4 a's in its calculation. But after looking up another resource, I can see that it really is linear. $\endgroup$ – rhombidodecahedron Jan 23 '17 at 18:50
  • $\begingroup$ Now that I am starting to understand this, however, I am a bit confused why it isn't appropriate for linear regression. I have uncertainties $\sigma_i$ for each $y_i$. Isn't it then appropriate for regression? $\endgroup$ – rhombidodecahedron Jan 23 '17 at 20:07
  • $\begingroup$ What exactly would be your independent variable in the regression? I don't see any dataset in your problem. You have values y, but no data x -- everywhere "x" appears is in an analytic integral. $\endgroup$ – jwimberley Jan 23 '17 at 20:12
  • 1
    $\begingroup$ I am able to evaluate each of those integrals numerically. Those would be the independent variables. $\endgroup$ – rhombidodecahedron Jan 23 '17 at 20:13
  • 1
    $\begingroup$ Ok. You should clarify your question then. If you are evaluating the data numerically, then I think my "linear algebra" formulation is linear regression in disguise. $\endgroup$ – jwimberley Jan 23 '17 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.