How can search engines exist in a capitalist system, then?

Because you don't propose a world without search engines, right?

Google tried for a long time to not have ads, but could not come up with an alternative revenue model and eventually had to.

You missunderstood. They charge money to promote, but they don't pay for content. If I open a decent magazine or newspaper, there are ads and there is content which paid for in some way. How is google so different? Googles payment is that they give you traffic for "free" if you play by their rules. I.e. in case of wikipedia I am sure they would have hefty debt if it was fair for both sides.

I see. Well, the problem with analogies is that they don't always fit. Another analogy is that Google is a magazine store, and present front pages to customers on the shelves. The store probably doesn't pay random magazines for showing up so much as they are a middleman in the sale between magazine and customer. (disclaimer: I've never owned a store, but while yes the store may take risks on unsold inventory, unless their deal with the magazine is "we send back unsold inventory")

Both Google and such a magazine store are middlemen do work to add value and take compensation for it.

In neither case is there any theft going on, nor is anyone getting a free ride.

I guess I also question the very premise of "free under condition X" is "free". Condition X is simply the payment. So you are getting Y for the price of X.

If you demand that Google charge for content then I do question how the very business model of search engine could work. Pretty much by definition if Google (or anyone else) has to pay to direct traffic to you then they are disincentivised to direct traffic to you. It's as if a store lost money on every sale, but made money if customers were present in the store.

If you think people are cynical now about Google wishing people clicking on promoted content (ads) then watch them race to the bottom where they (and Bing and everyone else) will only show promoted content.

Because here's really where your analogy breaks down: E.g. clicks on organic search results on Google brings money to the newspaper (via ads), but a news paper article author does not get paid per person reading their article. In other words: It may superficially be similar in actors and content creation, but all that matters in economics is incentives, and the incentives are completely different. And nobody will pay for a disincentive.