Meta/Yann LeCun were generating the hype around Galactica that caused expectations to be different. You can't say things like 'Type a text and Galactica will generate a paper with relevant references, formulas, and everything'[1] and expect your LLM not to make you look foolish. It wasn't about pushing the model to the limit, it was about setting realistic expectations of what it could do.

[1] https://statmodeling.stat.columbia.edu/2022/11/23/bigshot-ch...