Yeah. I'm not sure if anything with millions or billions of parameters will ever be "explainable" in the way we want.
I mean, imagine a regular multivariable function with billions of terms, written out on a (very big) whiteboard. Are we ever really going to understand why it produces the numbers it does?
KANs may have an order of magnitude fewer parameters, but the basic problem is still the same.
Personally I'm still basically with Geoff Hinton's early conjecture that people will have to choose whether they want a model that's easy to explain or one that actually works as well as it could.
I'd imagine the really big whiteboard would often be understandable in principle, but most people wouldn't be very satisfied at having the model go "Jolly good. Set aside the next 25 years in your calendar then, and tell me when you're ready to start on practicing the prerequisites!".
On the other hand, one might question how often we really understand something complex ostensibly "explained" to us, rather than just gloss over real understanding.
A lot of the time people seem to act as if they don't care about really knowing it, and just (hopefully!) want to get an inkling what's involved and make sure that the process could be demonstrated not to be seriously flawed.
The models are being held to standards that are typically not applied to people nor to most traditional software. But sure, there are also some real issues about reliability, trust and bureaucratic certifications.
> Are we ever really going to understand why it produces the numbers it does?
I would expect so, because we can categorize things hierarchically.
A medium-sized library contains many billions of words, but even with just a Dewey decimal system and a card catalog you could find information relatively quickly.
There's no inherent difficulty in understanding what a billion terms do, if you're able to just drill down using some basic hierarchies. It's just about finding the right algorithms to identify and describe the best set of hierarchies. Which is difficult, but there's no reason to think it won't be solvable in the near term.
I mean, imagine a regular multivariable function with billions of terms, written out on a (very big) whiteboard. Are we ever really going to understand why it produces the numbers it does?
KANs may have an order of magnitude fewer parameters, but the basic problem is still the same.