LLMs have already been connected to robotics. It's not that hard. The hardware is lagging the intelligence for sure though.

https://innermonologue.github.io/ https://ai.googleblog.com/2023/03/palm-e-embodied-multimodal... https://www.microsoft.com/en-us/research/group/autonomous-sy...