I thought this too, but it sounds like the performance of the GPU ultimately hol... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

_ea1k on July 11, 2023 | parent | context | favorite | on: GPT-4 details leaked?

I thought this too, but it sounds like the performance of the GPU ultimately holds it back. Maybe it is the software, but I have yet to see a test on Mac silicon that performed well.

schappim on July 11, 2023 [–]

It is not too bad, 7B is doing 432 tokens per second on a Macbook[1].

[1] Video: https://huggingface.co/datasets/huggingface/documentation-im...

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact