Howdy!
Instead of computing B-splines with the de Boor alogirthm (which is O(k*k) because of it's recursive fashion and memory-intensive), we can reformulate it as the k-th shifted ReLU power, as demonstrated in PowerMLPs, which is a lot easier to parallelize, has a much better time complexity and a smaller constant.
That could potentially make KANs as fast as MLPs, while maintaining their exact interpretability (the ReLU coefficients can be derived at both training time and compile time) and accuracy (it's not an approximation, it's mathematically equal).
Please do consider my proposal, as it could make KANs much more viable for traditional machine learning tasks (especially when combined with free knot KANs - another proposal for another day).
Thanks,
Alex <3
Howdy!
Instead of computing B-splines with the de Boor alogirthm (which is O(k*k) because of it's recursive fashion and memory-intensive), we can reformulate it as the k-th shifted ReLU power, as demonstrated in PowerMLPs, which is a lot easier to parallelize, has a much better time complexity and a smaller constant.
That could potentially make KANs as fast as MLPs, while maintaining their exact interpretability (the ReLU coefficients can be derived at both training time and compile time) and accuracy (it's not an approximation, it's mathematically equal).
Please do consider my proposal, as it could make KANs much more viable for traditional machine learning tasks (especially when combined with free knot KANs - another proposal for another day).
Thanks,
Alex <3