You're using them in reverse.
They are perfect for generating code according to your architectural and code design templete. Relying on them for architectural design is like picking your nose with a pair of scissors - yeah technically doable, but one slip and it all goes to hell.
Well, I have asked LLM to fix some piece of Python Django code so it uses pagination for the list of entities. And LLM came up with the working solution, impressively complicated piece of Django ORM code, which was totally needles, as Django ORM has Paginator class that does all the job without manual fetching pages, etc.
LLM sees pagination, it does pagination. After all LLM is an algorithm that calculates probability of the next word in a sequence of words, nothing less and nothing more. LLM does not think or feel, even though people believe in this saying thank you and using polite words like "please". LLM generates text on the base of what it was presented. That's why it will happily invent research that does not exist, create a review of a product that does not exist, invent a method that does not exist in a given programming language. And so on.
Im using them fine.
Im refuting the grandparent's point that they will replace basically all programming activities (including architecture) in 5 years.
