Depending on context, I cycle between espeak-ng with mbrola-en or RHVoice, but even plain espeak shouldn't be discarded.
RHVoice sounds slightly more natural in some cases, but one advantage of espeak-ng is that the text parsing logic is cleaner, by default.
For example, RHVoice likes to spell a lot regular text formatting. One example would be spelling " -- " as dash-dash instead of pausing between sentences. So while text sounds a little more natural, it's actually harder to understand in context unless the text is clean to begin with.
I don't know if speech-dispatcher does this for you, but I'm using a shell script and some regex rules to make the text cleaner for TTS which I don't need when using espeak-ng.
Another tradeoff: espeak-ng with the mbrola doesn't offer all the inflexion customization options you have with the "robotic-sounding" voices. When accelerating speech, these options make a qualitative difference in my experience.
However, I wasn't satisfied with the speech quality so now I'm using RHVoice. RHVoice seems to produce more natural/human-sounding output yo me.