I was thinking it would be fun just to have Siri use a Personal Voice, but your idea is better! And it could just offer to make your Personal Voice data available along with your Contact picture.
While that sounds like it would unlock some very cool experiences it also scares me to think about the potential abuses of making personalized voice models fairly easily available. It seems like the sort of thing that would need to stay secure on your own device. It would be great to see some kind of middle ground where a text to speech mechanism would generate audio output and send that, rather than make the model itself available.
