I've spent a lot of time trying to make Aider and Continue do something useful, mainly with Qwen coder models. In my experience they suck. Maybe they can produce some scaffolding and boilerplate but I already have deterministic tools for that.
Recently I've tried to make them figure out an algorithm that can chug through a list of strings and collect certain ones, grouping lines with one pattern under the last one with another pattern in a new list. They consistently fail, and not in ways that are obvious at a glance. Fixing the code manually takes longer than just writing the code.
Usually it compiles and runs, but does the wrong thing. Sometimes they screw up recursion and only collect one string. Sometimes they add code for collecting the strings that are supposed to be grouped but don't use it, and as of yet it's consistently wrong.
They also insist on generating obtuse and wrong regex, sometimes mixing PCRE with some other scheme in the same expression, unless I make threats. I'm not sure how other people manage to synthesise code they feel good about, maybe that's something only the remote models can do that I for sure won't send my commercial projects to.
Recently I've tried to make them figure out an algorithm that can chug through a list of strings and collect certain ones, grouping lines with one pattern under the last one with another pattern in a new list. They consistently fail, and not in ways that are obvious at a glance. Fixing the code manually takes longer than just writing the code.
Usually it compiles and runs, but does the wrong thing. Sometimes they screw up recursion and only collect one string. Sometimes they add code for collecting the strings that are supposed to be grouped but don't use it, and as of yet it's consistently wrong.
They also insist on generating obtuse and wrong regex, sometimes mixing PCRE with some other scheme in the same expression, unless I make threats. I'm not sure how other people manage to synthesise code they feel good about, maybe that's something only the remote models can do that I for sure won't send my commercial projects to.