In addition to all that you said about key management -- if you do manage to leak your keys and you chose a bad passphrase good luck proving to anyone that any correctly signed transactions were fraudulent/performed by someone else.