I've been experimenting with code-llama extensively on my laptop, and from my experience, it seems that these models are still in their early stages. I primarily utilize them through a Web UI, where they can successfully refactor code given an existing snippet. However, it's worth noting that they cannot currently analyze entire codebases or packages, refining them based on the most suitable solutions using the most appropriate algorithms. While these models offer assistance to some extent, there is room for improvement in their ability to handle more complex and comprehensive coding scenarios.
I think there is a decent chance SourceGraph will figure this all out. The most important thing at this point is figuring what context to feed. They can build up a nice graph of a codebase and I expect from there they can put in the best context and then boom.
They might also be able to train a model more intelligently by generating training data from said graphs.
I'm honestly failing to see the utility for LLMs, because the context for any given problem is far too small, and we're already at 33B parameter models. They just don't seem to be a technology that scales to an interesting problem size.
