Hi HN, after five months of hard work since Tabby's previous Show HN (
https://news.ycombinator.com/item?id=35470915), I'm thrilled to share you the release of Tabby v0.1.1.
Previously, Tabby ran exclusively on CUDA devices, posing a significant barrier for developers looking to effectively utilize LLMs in their day-to-day coding.
The Tabby team made a significant contribution by enhancing support for the StarCoder series models (1B/3B/7B) in llama.cpp. This enhancement allows these models to run on Metal, providing comparable performance to that of an NVIDIA GPU.
I eagerly look forward to receiving feedback from the community and witnessing the enhanced on-edge deployment experience!
References:
[1] Apple Installation Instructions: https://tabby.tabbyml.com/docs/installation/apple/
[2] Launch blog: https://tabby.tabbyml.com/blog/2023/09/18/release-0-1-1-meta...
