[0]: https://news.ycombinator.com/item?id=36882146

[1]: https://github.com/continuedev/continue/blob/33a0436193aa65a...

[0]: https://docs.sigstore.dev/signing/signing_with_blobs/

It seems it’s very hard to make good decisions about how to higher level things (modules, classes, dependency hierarchies etc.) without that context, and the programmer is forced to give the tool very specific instructions in order to compensate. At some point the instructions just need to be specific enough that you might as well be writing code again.

I have no doubt they'll get there eventually, but it seems like being able to write entire projects effectively might coincide with the arrival of true AGI. There's just so much context to consider.

Are they? I'd be interested to hear your experience on this. So far for me they have only really been able to summarise what I could find from the top few results searching online. They do a good job of summarising that, and might be quicker, but that's been it.

However, when I encounter an actually tricky issue, like a threading bug or a null pointer exception/type error sort of generic issue that's 5 levels removed from its source, these tools never manage it. Despite prompting saying I don't need a NullPointerException explained to me, figure out how this is null, the results are poor.

This might be my biases speaking, but it really does feel like I'm speaking to something that's good at transforming words and paragraphs into different formats but which has no actual understanding of the code.

Fixing tricky bugs often requires collecting additional information - stepping through code, looking at values of variables and making sure they are what you expect, etc. It's an iterative process and AI tools would need to be able to do the same thing - most humans wouldn't be able to solve errors like that just by looking at the code, and neither would an LLM.

I see the same issues with people who think AI is going to make scientific discoveries - it can't do that because making discoveries requires collecting data until something is certain or we have a clear picture. At that point, you don't need AI. AI won't be making discoveries until we can automate that entire process of forming a hypothesis, testing it / doing experiments, collecting data, refining your hypothesis, etc.

This has been my experience, except that the chat interface gets me to exactly the answer to my question considerably faster than a search engine.

I see these tools as search engines with much better user interfaces and customised responses.

I was hoping the GitHub or intelliJ integration of copilot would automate this, especially the latter has excellent static analysis of your code and could automatically provide the AI with relevant context, but they just don't.

Even when asking it to just annotate a function and specifically ask it to document any special cases or oddities it might have, I never got much more than e.g. updateFroobCache() annotated with "updates the froob cache". Wow thanks.

Yes, and eventually, one of us who is doing this will get tired of it enough to automate the process. May even earn them a few bucks.

Currently, you need to treat your LLM like it is a junior programmer. AI-coding tools and junior programmers will not give you the code you want if you don't write a detailed prompt. In my experience, however, AI coding tools will provide you with something closer to what you want than a junior programmer would.

Also, Apache Licensed.

Significant amount of work of course, seems polished.

There's also tracking:

"We track:

- the steps that are run and their parameters - whether you accept or reject suggestions (not the code itself) - the traceback when an error occurs - the name of your OS - the name of the default model you configured "

Looking at the doc strings generated in the video I don't see how these add any value to the code on screen. Surely there are better us-cases than that.

That really made me a freak out for a second, I started rechecking all the code I autocompleted with Copilot.

I recently tried it with "You are in a directory with a web app that does ... and you want to implement feature .... In each step, you can use ls or any other bash command and I will give you the output".

It was pretty hillarious how the LLM found its way around the codebase with ls, cat, find, grep awk and actually even managed to edit the code that way and do a commit.

Giving the LLM a bit better tools, like a version of "cat" that prefixes the lines with numbers, and a "swap" command that can do "swap 179 250 ..." to swap out the lines 179 to 250 with "..." would probably be enough to empower the LLM to be pretty efficient.

The next step might be to let the LLM manage its context window by allowing it to remove the last output with a command like "forget". So when the LLM does "cat somefile" and realizes that the output is not interesting, it can follow up with "forget" so the output will be replaced with "You deemed the output to be not interesting".

Those tools would probably evolve to make coding and managing the context window more and more efficient. Like "nicecat 100 200" to see the lines 100 to 200 with numbers prefixed. "keep 200 300" to forget the last output except lines 200 to 300 etc.

ed is the standard (LLM) editor

Say one of the files is talk.py and there is a function inside of it at line 17 which the LLM wants to change:

    17 def hello():
    18     print ("Hello!")

    swap talk.py 17 18
    def hello(name):
        print(f"Hello {name}!")

It's an interesting UX.

If you're using the OpenAI API, you can also automate this using the "tools"/"tool_calls" function call feature. https://platform.openai.com/docs/api-reference/chat/create

Please do write more. I'd love to replicate that. I keep coming across applications, where there is more "looking around the code" work than changing it. Even a slight help could mean a lot here!

This is a recipy for disaster imo, LLMs are not that good yet to give them the ability to act on their own

Can it integrate with Emacs or Vim?

If not, does anyone know of a similar project that does?

Nothing yet, but I've been playing around with a few models on vast.ai, and want to run them in my homelab soon. The low price of used 3090s and the increased performance and efficiency of new open source models makes this relatively accessible now.

> Also isn't the core product here the tuning of an LLM to this use-case, if so how does that work with self-hosting?

The core product here seems to be the ability to use any LLM inside an IDE. I've been avoiding proprietary LLMs for this purpose, so I'm interested in a solution that integrates with local models.

True, but this is not something this particular product would solve. There are already models specifically trained to work on code. What's appealing to me is the flexibility of being able to choose which one to use, rather than my workflow being tied to a specific product or company.

> the IDE integration seems to be the "easy bit"

I admittedly haven't researched this much, but this is not currently the case. There is no generic API for LLMs that IDEs can plug into, so all plugins must target a specific model. We ultimately need an equivalent of a Language Server Protocol for LLMs, and while such a project exists[1], it looks to be in its infancy, understandably so.

[1]: https://github.com/huggingface/llm-ls

I've had more luck with https://aider.chat although for whatever reason I haven't found it useful enough to actually use.

IMO, we are quite far from the capabilities of reliable issue -> PR generation yet even though it seems in reach.

For this reason, Dosu is focused on question answer, triaging, and generally context gathering because we see it as a pre-requisite to PR generation.

My only request for these tool-makers is: don't be shy. Tell me why I will stop paying for GitHub Copilot and pay for your tool. Make me sign up ASAP for a trial. Please don't pretend Copilot doesn't exist in the same universe. You probably have 5 seconds max (and I don't say this out of self-importance. I say this out of being a desperate little piece-of-shit who eliminates everything from their life that doesn't keep them on the path to their already less-than-ideal productivity :( I just can't afford that attention you demand)

Maybe its just me but chatting in a separate window... whats the point ? I can just open a tab in a browser with chatgpt.

What I want is something that understand my code base and can give me tipps of wich functions to use or where to find stuff and so on. Thats what cost me time.

Understanding someone else code or my old code. That takes time.

Writing code tests is what takes away from my time.

Writing a bad version of bubble sort is not what I want.

Especially for the free ones, just install one and try it for a day or two

Apache licensed.