Just remove one battery and it's off?

To be honest, I also have a Dymo label printer (very old model), and it does have a physical on/off switch, but even with the switch off it still consumes batteries like in 6 months.