You don’t need dependencies like webpack, react, or even any javascript at all to show static pages… and it would be a poor use of them just to write static html. But if you want interactive elements like image editors, rich text edition, fancy tables, then you start needing smarter and smarter build tools to bundle it all together and support the maintenance of interactive complex web applications.
I'd also argue that having a static build tool that generates HTML will also allow you to write more HTML and more complex HTML, as you can abstract away reusable components. So there are benefits even if you are serving non-interactive HTML pages. Just cache the React build output.
I don't use React-based static site generators, but I'd imagine if they become slow it's because it's generating a large code-base, but unless you provide some benchmarks showing compilation of React vs some other templating system I can't really take your argument against it for this use case.
The developer experience is better. I personally like type safe templates, and just the JSX bits of React are one of the most mature type safe templating frameworks with very good editor support.
I'd also argue that having a static build tool that generates HTML will also allow you to write more HTML and more complex HTML, as you can abstract away reusable components. So there are benefits even if you are serving non-interactive HTML pages. Just cache the React build output.