0
$\begingroup$

Can someone help me understand how Kolmogorov-Arnold Networks (KANs) make use of B-splines? They talk about spline grids in the paper without explicitly defining what they mean by it AFAIK. For instance they say:

Update of spline grids. We update each grid on the fly according to its input activations, to address the issue that splines are defined on bounded regions but activation values can evolve out of the fixed region during training.

  • Do KANs keep track of input activation statistics in every batch?

  • Do they use mini-batches or do they simply use the whole dataset?

  • How does changing input activations change the splines?

Relevant equations:

$$\phi(x) = w_b b(x) + w_s \text{spline}(x)$$

$$b(x) = \text{silu}(x) = \frac{x}{1 + e^{-x}}$$

$$\text{spline}(x) = \sum_i c_i B_i(x)$$

If you could help me under the underlying idea behind B-splines and basis functions / spline grids, that would be very nice! I am interested in the details.

New contributor
Transcendental is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.