For most illnesses, you are correct, reducing a fever is unnecessary. However infants and toddlers are at risk for febrile seizures should their temperature increase too high, too quickly. In those scenarios (fast rising temp) there is considerable medical benefit to intervention.
What you write is commonly believed, but not actually true. There is no medical benefit to intervention:
"The most consistently identified serious concern of caregivers and health care providers is that high fevers, if left untreated, are associated with seizures, brain damage, and death. It is argued that by creating undue concern over these presumed risks of fever, for which there is no clearly established relationship,"
