You can have an electronic circuit that is open, unless you press a button, then it's powered on and keeps power until a timeout/event. I know, it requires some more complex concepts than a bidirectional physical switch, but it's doable.

The twitter thread complains about controllers which empty their battery between uses because they keep trying to connect; but it's not something that's mandatory: an at-rest device can stay at rest without user interaction. I've used hardware on which the battery was still usable after years without use; it does requires very careful design, though.