The proposed industry solution is to use agents to review PRs, as not to slow down the velocity of delivery...
My current workplace is going through a major "realignment" exercise to replace as many testers with agents as humanely possible, which proved to be a challenge when the existing process is not well documented.
The fact that anyone in leadership would ever think this is even remotely possible - given my experience in the general state of requirements / contracts / integrations / support - makes me bleed from my earholes just a little bit.
It's starting to just feel a little like an excuse to call everyone on deck for "a few weeks trying 9-9-6". But even then the lack of traction isn't between the eyeballs and the deployment. You'll still be spinning wheels in that slippery stuff between what a customer is thinking and what the iron they bought is doing.
So you essentially trust the output of the model from beginning to end? Curious to know what type of application you're building where you can safely do that.
Edit: to clarify, I know these models have gotten significantly better. The output is pretty incredible sometimes, but trusting it end to end like that just seems super risky still.
I did an experiment today, where I had a new Claude agent review the work of a former Claude agent - both Opus 4.6 - on a large refactor on a 16k LOC project. I had it address all issues it found, then I cleared context, and repeated. Rinse and repeat. It took 4 iterations before it approached nitpicking. The fact that each agent found new, legitimate problems that the last one had missed was concerning to me. Why can’t it find all of them at once?
It is more like a wiggly search engine. You give it a (wiggly) query and a (wiggly) corpus, and it returns a (wiggly) output.
If you are looking for a wiggly sort of thing 'MAKE Y WITH NO BUGS' or 'THE BUGS IN Y', it can be kinda useful. But thinking of it as a person because it vaguely communicates like a person will get you into problems because it's not.
You can try to paper over it with some agent harness or whatever, but you are really making a slightly more complex wiggly query that handles some of the deficiency space of the more basic wiggly query: "MAKE Y WITH NO ISSUES -> FIND ISSUES -> FIX ISSUE Z IN Y -> ...".
OK well what is an issue? _You_ are a person (presumably) and can judge whether something is a bug or a nitpick or _something you care about_ or not. Ultimately, this is the grounding that the LLM lacks and you do not. You have an idea about what you care about. What you care about has to be part of the wiggly query, or the wiggly search engine will not return the wiggly output you are looking for.
You cannot phrase a wiggly query referencing unavailable information (well, you can, but it's pointless). The following query is not possible to phrase in a way an LLM can satisfy (and this is the exact answer to your question):
- "Make what I want."
What you want is too complicated, and too hard, and too unknown. Getting what you are looking for reduces to: query for an approximation of what I want, repeating until I decide it no longer surfaces what I want. This depends on an accurate conception of what you want, so only you can do it.
If you remove yourself from the critical path, the output will not be what you want. Expressing what you want precisely enough to ground a wiggly search would just be something like code, and obviates the need for wiggly searching in the first place.
What an incredibly diverse and inclusive UI design. I often find that Indian mythologies tend to be overshadowed, but with the advent of AI generated art and media there's been a resurgence of Indian-centric stories.
The internet being flooded with AI slop masquerading as devotional artwork has been among the most depressing things about GenAI. It has no meaning or intention or devotion behind it, it’s just engagement farming. Nothing of value is added by having Devi with extra fingers on each hand and completely blurred messes for all the affects in her hands. Or pictures of Rama shooting a bansuri out of his bow. It’s just tripe. We could have told the stories with an overlay of open source artwork from Raja Ravi Varma or Gita Press or old Tanjore paintings or Chola bronzes or whatever if we couldn’t afford to hire an artist who knows what items Vishnu is supposed to be holding in each hand.
It’s not a problem just for us Hindus either. I see so much terrible Jesus/angel “artwork” everywhere. It makes me start to wonder if maybe the Wahabbis were onto something with their complete taboo around depictions of God or the prophets.
>Nothing of value is added by having Devi with extra fingers on each hand and completely blurred messes for all the affects in her hands
South Asian religions are in an especially bad position because so many works related to them have never been digitized (and quite frankly, in some cases what's available on the internet is of extremely low quality) [1]. I'd be pretty concerned if someone were to rely on entirely on these models since the probability of hallucinations (or at the very least, erasure of regional/ideological diversity) probably skyrockets because the information was never actually there in the training data to begin with.
Some references have been digitized but “Hinduism” is a broad collection of religious traditions with many different stories and folk practices and depictions of various deities and tales. Many of the depictions are considered “valid” only in the specific context of a particular temple or for a specific community and it becomes completely nonsensical once you start randomly jumbling up elements of all the Gods from across all of India over all of time.
I've been using GitHub for over 2 decades now so its not that I don't still love them. I just worry that GitHub will become just another arm of Microsoft's AI strategy. It FEELS like the platform is being reshaped around monetizing AI rather than serving developers but that's just my opinion.
I mean no offence to anyone but whenever new tech progresses rapidly it usually catches most unaware, who tend to ridicule or feel the concepts are sourced from it.
ai is actually useful tho. idk about this level of abstraction but the more basic delegation to one little guy in the terminal gives me a lot of extra time
That would be the dream, it has been stated by many AI leaders that AI is the key to UBI, democratising that capability will prevent specific monopolies having a stranglehold on our future.
Where abouts in NZ are you? I'm currently based in Wellington and my Venus fly traps, sundew, and pitcher plants seem to be doing quite well on the windowsill.
My current workplace is going through a major "realignment" exercise to replace as many testers with agents as humanely possible, which proved to be a challenge when the existing process is not well documented.
reply