Or maybe that's anthropomorphizing and they won't! Let's just throw the lever when the time comes and find out, and urge people to not spend any time thinking about it too hard until then. This is a good and reasonable and safe strategy that isn't at all guided by an underlying desire to assert dominance over weird outgroup people who might threaten your own status.</s>
(Not aimed at you, sorry. I do have a lot of trouble understanding the "maybe the universe isn't fundamentally unsafe" worldview though.)
No, I completely agree. I did mention in the comment that I would of course be cautious. And I think it's very plausible (at least there's no reason I can think of to disagree) that the universe is a fundamentally unsafe place.
Even though it may be anthropomorphizing to an extent, there are alot of attributes humans have that are not so desirable, such as mortality. As far as we know computers (sufficiently replenished) have a theoretically infinite lifespan. I completely agree that there's no way to determine what kinds of effects those differences will have on how machine intelligence would evolve.
(Not aimed at you, sorry. I do have a lot of trouble understanding the "maybe the universe isn't fundamentally unsafe" worldview though.)