I was using my phone for contactless pay but stopped during the pandemic because I realized it was more cumbersome than using a credit card to do the same thing. The phone tries to scan my face, fails because of the mask, then just waits for a hot second to see if my face might change shape, then prompts for a password. With the credit card, I just swipe it over the reader and I'm done.
I use my Watch, which is almost always simpler even without a mask.
However, you should be able to hurry it along to prompt for a password. Apple doesn’t make it easily discoverable, but if you tap on the status message when it’s trying to decide whether it recognizes you, it will prompt you for a password instead (while logging in, and I imagine during Apple Pay as well).
You can configure your iPhone not to use FaceID for Apple Pay only. That's what I did for the same reason as you. I just quickly enter my PIN to use Apple Pay.