Thanks for the tip with ollama

If you do:

1. ollama run deepseek-coder:6.7b

2. pip install litellm

3. litellm --model deepseek-coder:6.7b

You will have a local OpenAI compatible API for it.

ollama is actually not a great way to run these models as it makes it difficult to change server parameters and doesn't use `mlock` to keep the models in memory.

What do you suggest?

vanilla llama.cpp (run a `/server`)