If I'm not mistaken, P.Eng are merely needed the rubber stamping, not to do the grunt work. I doubt they'll go through all the specs, re-run all the simulations, and verify all the parameters.

Also, where is your P.Eng be any useful to justify the learning coefficients of your AI, or to protect about the cosmic ray bit-flip in the hardware your proven correct software is running on ?

Engineering for safety is about testing, not manufacturing quality. Or course you need to build quality in order to pass tests, but perfection is not a goal of any component. Tests measure quality, and failsafes and control layers reduce failure rates.

Thanks for the Therac-25 reference :)