Im not sure this is true, the part in the cable is just tiny pins and far more vulnerable than the wafer. For years now I've dug woodchips and other junk out of my phones usb-c socket with paperclips and never came close to damage. It seems sufficiently recessed to prevent all but the most aggressive attack.
Edit: in fact, trying now it seems to be perfectly recessed that I can't even push on it from an angle with the connector, it has to go straight in a bit before it even touches, on any of my devices
I mean, I'm referring to things I've had to deal with. Failure isn't obvious until the user complains of random disconnects, and then you find that although everything looks fine from the outside, you can wiggle the wafer inside like a loose tooth. The plastic of the wafer has cracked and is being held in place only by the metal in the contacts.
I'll grant that it's robust in that it continues to work more than I'd expect, for example continuing to charge the laptop but not reliable for the external display. And once connected, sure it's secure.
I suspect it's not the cable mating that causes the failure, but other foreign objects… keys in the bag or something… that manage to dislodge it. And some laptops have more play in the connector than others, which I think allows the connector on the cable end to hit the wafer with an angle of attack sufficient to dislodge it. Some of my users handle heavy bulk materials all day and are anything but gentle with computers.
In any case, I'm sure the designers considered this, and far be it from me to second-guess their compromises. It is what it is. But if the port has failure modes, this is one of them.
It's almost as if the designers have considered this potential failure mode for devices that often get dropped even while plugged into a charger :)
I can emphasize with many of the gripes people have with USB-C cables and supported modes being confusing, but the port is really, really well thought out: Make the cable the part that fails more easily [1]; make it so that the plug only minimally acts as a lever while partially or fully plugged in; cover the pins on the plug (because otherwise people will inevitably touch them, and these cables can carry up to 48 V of voltage).
[1] That part has actually worked well for everything but my Yubikey :( USB-A and an adapter it is for the next one.
A significant issue probably affecting your Yubikey is that there are very few male USB-C plugs available. The ones that do exist are mostly designed for USB-C cables, which uses a tiny 0.8mm thick "paddle card" fully embedded in the plastic cover.
Alternatively, there are "brick nogging" connectors which stand up straight from the PCB and are therefore unusable for a Yubikey, or "sink board" ones which require a custom jig for assembly.
Meanwhile with USB-A you don't even need a male connector at all - just have a bit of your PCB stuck out in a vaguely USB-A-looking shape and it'll work just fine.
Edit: in fact, trying now it seems to be perfectly recessed that I can't even push on it from an angle with the connector, it has to go straight in a bit before it even touches, on any of my devices