Thanks! So it looks like AuthorizedPrincipalsFile/AuthorizedPrincipalsCommand gives us a method for doing this. This would have to be combined with some sort of user ID management system still, like distribution of /etc/{passwd,group} files, LDAP/AD, etc.
And for the ssh ca part, bless and teleport (as others have mentioned).
There's the option of putting stuff in ad/ldap - but if you're already using ad, kerberized ssh (and sudo etc) might be the way to go.
I like the idea of a system that's simpler than ad/ldap+kerberos - and ssh certs fits most of the bill.
The challenge becomes auth/authz beyond just login - ldap basically requires ssl ca anyway - and at that point, especially with kerberos set up - I think one might be better off sticking with one complex auth/authz system rather than two...