I’ve seen neural nets combined with decision trees. There’s a few ways to do such hybrids. One style essentially uses the accurate, GPU-trained networks to push the interpretable networks to higher accuracy.

Do any of you think that can be done cost-effectively with KAN’s? Especially using pre-trained, language models like LlaMa-3 to train the interpretable models?