This is where artifact binding can greatly increase the security….

Browser sends artifact to RP, RP fetches assertion from IdP via HTTPs, afterwards verifies the signature.

Signature verification is not implemented correctly? The attacker still needs to break HTTPS…. And then you would have a big problem anyway.

A common pattern of SAML vulnerability is one that allows logins from one org access to resources of another org, which is not mitigated by trusting IdP's.