Conversely, if there are so many prisoners or the disenfranchised that, in a democracy, they can sway elections and drive policy, isn't that a sign there's something wrong with the system, e.g. disenfranchisement and prison is being used as a weapon against parts of the population? Denying them the vote means the system is that much harder to fix.
On the other hand, if only those who really deserve it lose their right to vote (i.e. a small fraction of the population), then it doesn't really matter if they can vote or not - their small numbers mean they won't make a difference. But the benefit is that disenfranchisement can no longer be used as a political weapon.
You shouldn't be able to vote if you are actively in prison on the day of the election. Or maybe cases like house arrest. But if we say that you're able to be walking around in society with us, including probation/parole, you should have all the rights that others have.
On the other hand, if only those who really deserve it lose their right to vote (i.e. a small fraction of the population), then it doesn't really matter if they can vote or not - their small numbers mean they won't make a difference. But the benefit is that disenfranchisement can no longer be used as a political weapon.