There's no difference between trusting the server to deliver Javascript cryptography source code and trusting the server with your secrets. The server can just deliver code (in a number of different ways) to compromise those secrets.
My understanding from the Apple/FBI fiasco is that it's an open question whether the US government is legally able to compel you to write and deliver custom code to your clients. But any local judge can issue you a warrant to hand over encrypted customer data that you're holding the key to.
But IANAL, maybe my take on that is overly simplistic.
Actually, Filekit is meant to be integrated into an application, and that application is responsible for delivering JS and handling identities, not the Tanker server.
It only contains public keys and encrypted DEKs (data encryption keys); see this image: https://docs.tanker.io/filekit/latest/going-further/file-enc...
The file content should therefore only be accessible by holders of private keys for which the file was encrypted.