Maybe a significant percentage of ad clicks are fraudulent and "more sophisticated" technologies like https break malicious traffic? I couldn't come up with alternative plausible explanation.
I believe this situation also shows weird dynamics with fighting adfraud. In general too much fraud can crash ad click economy, but on micro scale a lot of actors indirectly benefit from it so they don't have incentive to fight with it strongly. Especially if most automated way of fighting bots will also affect legitimate traffic (e.g. captcha for real users).
I would say real world analogy is oil prices and OPEC. Or more classical analogy is prisoner's dilemma. Individual actor incentives are different than best solution for the whole ecosystem.
The real cause is downgrade protection. When your browser loads an HTTPS page, it will refuse to load (or at least warn when loading) any other resources over HTTP: js, css, iframes, etc.
This is to ensure that the icon you see in your URL bar is actually accurate: if a page loads over HTTPS, but consists entirely of a single HTTP iframe, that nice green lock is totally meaningless.
So when the OP switched their site to HTTPS-only, they lost the ability to ever display any from HTTP-only bidders. So the set of people bidding on their ad slots went down, the price went down, and their revenue went down.
That is an excellent and counterintuitive but logically sound market-response explanation for the behavior.
I'm trying to think of how Google can attempt remedying this, with the more obvious options being to either increase the non-HTTPS search penalty, or perhaps to buy up some of the ad stock itself.
I believe this situation also shows weird dynamics with fighting adfraud. In general too much fraud can crash ad click economy, but on micro scale a lot of actors indirectly benefit from it so they don't have incentive to fight with it strongly. Especially if most automated way of fighting bots will also affect legitimate traffic (e.g. captcha for real users).
I would say real world analogy is oil prices and OPEC. Or more classical analogy is prisoner's dilemma. Individual actor incentives are different than best solution for the whole ecosystem.