Going passwordless with long-lived sessions requires more complex session management though. If you don't time-out sessions then you increase the cumulative probability of a live session eventually being hijacked through XSS, MITM (coffee shop, rogue wifi), or malware etc.
And if you want the user to be able to perform sensitive operations (edit their personal details for example) then you'll have to ask for a OTP or email verification every time. These methods tend to be higher friction than a password box.
I'm not sure I see the XSS vuln, or rather, a site might have an XSS vuln and long sessions would make it worse, but I don't see long sessions causing XSS. MitM would be possible without TLS, but not with it. Malware is always a threat, but if it can read cookies it might be able to read cached passwords etc. too.
Sure you don't want to constantly bug the user but not every site needs to do that. Especially for sporadically-used sites, "receiving email" could be less of a pain than keeping track of passwords.
A session can be long-lived without being indefinite. We might decide that any authenticated site visit within the last week is new enough not to repeat the passwordless process, or we might say two weeks or a month or whatever.
And if you want the user to be able to perform sensitive operations (edit their personal details for example) then you'll have to ask for a OTP or email verification every time. These methods tend to be higher friction than a password box.