_Could_ they have done the same thing with a tiny fraction of the money? Grok 3 benchmarks are SOTA for both base model and reasoning. By definition, nobody has been able to do the same thing with any amount of money (discounting o3 which has been teased but is unreleased). That may change in the future! But as of now this is the case.
So apart from the part where SOTA doesn't mean anything in the real world (there is no monetisation, there's no moat), please, it's benchmarks, we all know how you beat those since 2023.
Why don't you do it then? If you are talking about Deepseek "$5M", then you would be interested to know that they pay 7 digit salaries and reportedly have H100s worth $2B[1].
Just wonder if it matters? If Google spent 10x as much in the first 5 years of its life would it be a worse company now? Giant TAM, winner takes all (or most?), all that matters is winning.
People like Demis Hasabis and Derio Amodei say that R1 efficiency gains are exaggerated. $5M training cost seems to be fake as sources suggest they own more GPUs.
