Yes your understanding is how almost everyone treats the word today. The slur is from a different meaning & different context. You'd never come across it in regular life, unless you were, like, into the history of baseball:

https://en.wikipedia.org/wiki/Dummy_(nickname)