Because the real world is more complex than a lever, and there are many unknowns in how either physical or human manifestations respond to actions, and doing nothing removes your intent from the mix.
A very clear example of this is medicine with the "do no harm" principle - that the actions of the physician should be chosen to minimize the scenarios of harming the patient under any circumstance - under chance, under lack of patient compliance, etc.
Furthermore, the actions in the real world have also different experiences and meanings. Its easy to think about pulling a lever to kill 1 instead of 5, but not easy to think of killing one to harvest their organs and save other 5, though they are, with a lot of abstraction, "equivalent moral actions".
A very clear example of this is medicine with the "do no harm" principle - that the actions of the physician should be chosen to minimize the scenarios of harming the patient under any circumstance - under chance, under lack of patient compliance, etc.
Furthermore, the actions in the real world have also different experiences and meanings. Its easy to think about pulling a lever to kill 1 instead of 5, but not easy to think of killing one to harvest their organs and save other 5, though they are, with a lot of abstraction, "equivalent moral actions".