There's another problem here that I haven't seen anyone talking about, and that's the futility of trying to distinguish between "good bots" and "bad bots".
The idea of Anubis is to stop bots that are meant to gather data for AI purposes. But you know who has a really big AI right now? Google. And you know who are the people who have the most bots indexing the web for their search engine? Yup, Google.
All these discussions have been assuming that Googlebot is a "good bot", but what exactly is stopping Google from using the data from Googlebot to feed Gemini? After all, nobody's going to block Googlebot, for obvious reasons.
At most, surely the only thing that blocking AI bots will do is stop locally-running bots, or stop OpenAI (because they don't have any other legitimate reason to be running bots over the web).
The idea of Anubis is to stop bots that are meant to gather data for AI purposes. But you know who has a really big AI right now? Google. And you know who are the people who have the most bots indexing the web for their search engine? Yup, Google.
All these discussions have been assuming that Googlebot is a "good bot", but what exactly is stopping Google from using the data from Googlebot to feed Gemini? After all, nobody's going to block Googlebot, for obvious reasons.
At most, surely the only thing that blocking AI bots will do is stop locally-running bots, or stop OpenAI (because they don't have any other legitimate reason to be running bots over the web).