CPUs are quietly becoming very well-balanced machines for BS 1 inference. The la... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		pclmulqdq 4 months ago \| parent \| context \| favorite \| on: Why DeepSeek is cheap at scale but expensive to ru... CPUs are quietly becoming very well-balanced machines for BS 1 inference. The latest Intel Xeons should be at ~20 TPS.

Spooky23 4 months ago [–]

A base Mac Mini is ~20 :)

pclmulqdq 4 months ago | [–]

Oh yeah, I did that math not assuming any quantization. I think if you can get a 3-4 bit quant working + int8 math, ~80 might be achievable.

Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact