It's also a little insane to me that what Adept has been supposedly building for years with 300+ mil in funding can now be built in a day with Open AI APIs?

I think Adept pivoted along the way but original concept was very similar to this.

Agreed! This is part of the motivation behind https://github.com/OpenAdaptAI/OpenAdapt

But its too expensive to become practical with the OpenAI API. Also, demo is cool until you see the real-world webpages, then you'll realize that this only works less than %50 of webpages.

GPT-4V may be surprisingly robust here. Set of mark prompting(which is accomplished here with Vim) improves grounding by a silly high amount. https://som-gpt4v.github.io/

https://www.adept.ai/blog/experiments :)

Yep, took inspiration from them and a couple other startups

What other startups did you use for inspiration?

This is precisely the demo I am thinking.