There are multiple branches they are exploring. This is a more structured one. But they also work on Agents that load the website and produce clicks to do the task. Also, this requires hand design, but they also work on generating the gui just-in-time, based on context.
They also have this new design gui for visual programming of agents, with boxes and arrows.
It's going to be a hybrid of all these. Obviously the more explicit work done for interoperability, the easier it is, but the gaps can be bridged with the common sense of the AI at the expense of more time and compute. It's like, a self driving car can detect red lights and speed limit signs via cameras but if there are structured signals in smart infrastructure, then it's simpler and better.
But it's always interesting to see this dance between unstructured and structured. Apparently any time one gets big, the other is needed. When theres tons of structured code, we want AI common sense to cut through it because even if it's structured, it's messy and too complicated. So we generate the code. Now if we have natural language code generators we want to impose structure onto how they work, which we express in markup languages, then small scripts, then large scripts that are too complex and have too much boilerplate so we need AI to generate it from natural language etc etc
They also have this new design gui for visual programming of agents, with boxes and arrows.
It's going to be a hybrid of all these. Obviously the more explicit work done for interoperability, the easier it is, but the gaps can be bridged with the common sense of the AI at the expense of more time and compute. It's like, a self driving car can detect red lights and speed limit signs via cameras but if there are structured signals in smart infrastructure, then it's simpler and better.
But it's always interesting to see this dance between unstructured and structured. Apparently any time one gets big, the other is needed. When theres tons of structured code, we want AI common sense to cut through it because even if it's structured, it's messy and too complicated. So we generate the code. Now if we have natural language code generators we want to impose structure onto how they work, which we express in markup languages, then small scripts, then large scripts that are too complex and have too much boilerplate so we need AI to generate it from natural language etc etc