Apparently Google is capable of doing cool things. DeepMind’s speculative sampling achieves 2–2.5x decoding speedups in LLM. That brings cost down significantly, without degradation in quality.