The remote does work. This is an added convenience. To be clear, the remote continues to work. At no point does the remote stop working. If you use your phone to cast, the neighbour can still use the remote to pause. If you don't give the neighbour the SSID, he can still use the remote. Your neighbour does not need to be on your network. If he's on your network the remote and his phone will work. If he's off your network, the remote will work. The remote will not stop working. Hope that clarifies.
The original argument was in favor of the OG Chromecast's method, wherein: There is no dedicated remote laying around that anyone can pick up and unambiguously make work.
It is good to have options.
(And modern Google-produced streaming devices, whether they are good or bad, do provide options.)
I might as well publish the SSID and passphrase on a billboard, or just go full Bruce Schneier and leave it open for all.
(Or.... You know, a real remote! With real buttons!)