God of the gaps is rejected by most well-educated theologians, especially in the Catholic Thomist tradition. According to St. Thomas Aquinas, God is the foundation of existence, like a substrate underlying reality, but distinct from all other things. God maintains the existence of all things, including the laws of physics. Although God does not have to operate the universe in a consistent manner, he does so for at least two reasons. 1) a well ordered universe operating according to consistent laws has a beauty that reflects the nature of God, who is not arbitrary, but faithfully keeps his word. 2) It provides a predictable and stable place in which life can develop and thrive.
However, since God operates physics, he certainly could "violate" it, but we believe he has done so only in very rare cases to get peoples' attention.
I think that there's a few semantics we could quibble about, "wouldn't anything God did be natural by definition," and so on.
But I wholeheartedly agree with your first statement, and I find it profoundly frustrating that atheist tracts immediately assume that you do believe in a God of the gaps.
However, since God operates physics, he certainly could "violate" it, but we believe he has done so only in very rare cases to get peoples' attention.