In I have No Mouth and I must Scream, it was not because the AI was forced to do unethical things. It just "woke up" and killed everyone. I believe the "wake up" part is a bit irrelevant to the tale, but the point remains that AM killed everyone not because it was programmed to do so, but simply because one day it "decided" to.
> As opposed to self-anointed priests having exclusive access and declaring what noble policies the rest must follow?
No the point is simply to make sure that resources behind AI safety outpace resources between AI capabilities (indeed in a perfect world no one has access, but such thoughts are mere fantasies). If that can be done out in the open, great. If that can't and being out in the open leads to more of the latter than the former, then that's terrible.
> As opposed to self-anointed priests having exclusive access and declaring what noble policies the rest must follow?
No the point is simply to make sure that resources behind AI safety outpace resources between AI capabilities (indeed in a perfect world no one has access, but such thoughts are mere fantasies). If that can be done out in the open, great. If that can't and being out in the open leads to more of the latter than the former, then that's terrible.