The PAKE algorithm lets you spend an interactive roundtrip to buy a full-strength key out of a weak shared secret. An attacker can attempt to guess the passphrase, and their chances are non-zero (one out of 65536 with the default configuration), but when they guess wrong, the whole protocol shuts down, and the real participants have to start over again, with a new code. So the only way for the attacker to win is for you to restart over and over again until they get lucky. Kinda self-limiting that way :).
An attack on the PAKE would involve the attacker seeing the secret value as it was transferred to the recipient and then beating the recipient to the handshake. So there is security value in being prompt in putting in the secret value at the receiving end.
That is as opposed to sending a public key or key fingerprint. In that case there would be little value to the attacker in seeing the transfer. They would have to MITM the transfer of the key itself. If you wanted to prevent the attacker from sending bogus files you would also have to transfer some sort of signing key.
So a short, time limited, secret vs a longer public value.
https://github.com/magic-wormhole/magic-wormhole/blob/master... has a larger writeup.