I agree. As a web publisher who depends on advertising for revenue and Google for traffic, things are getting more and more difficult. Google providing answers in the search box robs legitimate publishers of traffic. But the industry is so fragmented that a concerted legal action against Google is not possible.
Since this is HN, I expect people are going to start chanting "Die you ad supported publishers or find a new business model. We love Google because it's great for the users to read the answer on Google and not have to visit your ad laden site". I understand this sentiment but don't agree with it. Users love anything that's free. That does not mean Google can steal it and give it to them.
The implicit contract between publishers and Google is that publishers let Google crawl and index their content. In return, Google includes the publisher in their search results so users can find the information they are looking for and publishers get traffic. Outright copying of this information is stealing because it robs publishers of their revenue stream.
OK, so now that implicit contract changed and says Google can show that information above the organic search results. Given this, publishers do have a choice of blocking Google in their robots.txt.
this is an implicit agreement (not explicit). the agreement changed. publishers still have a chance to respond... dont like it? block google! (its just 1 line in robots.txt) go rely on word of mouth, bing, ads and emailing to get traffic!
dont want to do that? oh i see... well that means you voluntarily agree to the agreement and have no right to complain.
> you voluntarily agree to the agreement and have no right to complain
See, that sounds exactly like the kind of thing a standover merchant would say: feel free not to pay the protection fees, sure would be a shame if something would happen to your organic search traffic…
Blocking Google via robots.txt isn't an option because Google has a monopoly. You're amputating the whole leg because of an infected toe. Instead of free samples at Costco you're giving away entire boxes and users love you for it because it's free. It's the producer who loses the revenue.
At what point do you draw the line? Is it OK if Google steals 1 line? How about a whole list of steps for how to do something? How about when Google takes an image from your page and puts it in its hero search result "answer box" but with a link to another site?
As a publisher, I feel the right solution is that the publisher decides the title and meta description -- which are the 2 things Google needs to show in their SERPs. If my title and description snippet are clickbaity or not good enough, other publishers can win. That's fine because publishers are competing on a level playing field.
Web publishers have gotten lazy. Instead of becoming more dependant on Google and then complaining about being marginalized they should put more effort into marketing directly to their target audience and finding non-advertising revenue models.
Even when publishers have a good brand, it doesn't change user behavior. How often do users directly go to Wikipedia to look something up? Google search is integrated into Chrome, Android, Google Home. Google is a monopoly.
Yes, publishers should find non-advertising revenue models. Yes, they should find non-Google sources of traffic. But that's blaming the victim. The fact is that Google has started stealing more and more content from large and small publishers in order to keep the user from leaving Google. And it's not OK for a monopoly to do that.
As has been pointed out, so many users have gotten in the habit of using an omni-box to go to all URLs via a search. Staying its a choice to block Google is a false choice as that would mean death. The analogies are spot on. Google is in a position of power due to search dominance and content creators have very little unless every content creator decided to all drop off together. This is unlikely.
Since this is HN, I expect people are going to start chanting "Die you ad supported publishers or find a new business model. We love Google because it's great for the users to read the answer on Google and not have to visit your ad laden site". I understand this sentiment but don't agree with it. Users love anything that's free. That does not mean Google can steal it and give it to them.
The implicit contract between publishers and Google is that publishers let Google crawl and index their content. In return, Google includes the publisher in their search results so users can find the information they are looking for and publishers get traffic. Outright copying of this information is stealing because it robs publishers of their revenue stream.