I believe that on-device models will become a significant trend. I previously released a local file manager app called Riffo, and we have been using the OpenAI API since its launch. However, our users have informed us that they need on-device models. This indicates to me that it's an important issue.

We have been developing an on-device model recently. For macOS, we are using Ollama to host the on-device model on Mac. We have not yet found a perfect solution for the Windows version. Thank you for sharing Llamafile; we might test it for the Windows on-device model.