This is only scratching the surface of the question.
For interest, there's a very common negative expected value bet that almost everyone is required to make: insurance.
We don't consider that gambling, in fact we often tell our parents to buy some when they fly on holiday.
Why? The answer touches on the lottery.
We care about not just the average case, we care about what might happen.
Regarding Kelly criterion, there's a good reason why people don't used exactly the amount it says. If you look at the risk, ie the chance your probability is wrong, there's a chance you are overbetting.
I've seen this sentiment alot and i believe it's mostly right but it depends on who you are. There is no 'Best investment Strategy' for everyone.
Should a 20-something invest all his savings into crypto? Sure. Should a 45 year old with kids? hell no.
If the 20-something losses all their savings, that sucks. If a 45 year old losses all their savings they have people to provide for. Which doesn't just suck, it's detrimental to his life and hapiness.
A 20 something has 40 years to bounce back, a 45 year old has 15. Time horizions dictate risks that can be taken.
But there is a best investment strategy: the one that maximises growth.
A constant-fraction rebalanced portfolio has nothing to do with avoiding loss. It's purely about maximising growth. Such a portfolio, in the long run, outperforms all individual assets it is constructed from.
I agree with your general sentiment. My reasoning to get there is different:
I don't, for example, think anyone should invest all their capital into a risky asset. Not because it might crash, but because it performs poorly compared to the best investment (which is a balance weighted toward safe assets.)
Logarithmic utility corresponds to maximum growth of wealth (Kelly criterion), so insurance is actually often compatible with maximum growth of wealth.
How can something be negative EV yet maximise growth? Compound returns.
Insurance is only negative EV when considering a single period. The ongoing act of having insurance is positive EV in terms of growth. The way to get to that is to count EV as the geometric mean instead of arithmetic mean.
This is a very common mistake still, even though it was discovered by Bernoulli in 1734. I strongly recommend reading that paper. It is very easy to read.
If you assume a well-functioning insurance market, I think the argument can be made that the negative expected value should however be very small. Then going one step further, I think mitigating downside potential is actually the opposite of gambling. Driving without insurance is gambling.
what does "well functioning" mean in this case? The insurance market's profits literally are the negative expected value. If the negative expected value is very small, there are very small profits for the insurance companies. They would only do that if there was pressure on them re competition, which is the opposite of what is happening nearly everywhere in almost every market.
High competition low profits would be well functioning for a consumer, low competition high profits would be well functioning for an investor
I think that for the serious cases of unexpected misfortunes, an insurance is a compressor where all population events are the whole signal and individuals make the peaks (well sometimes many a person are in the same event):
For the subset of people that would need an insurance without knowledge that could prevent that, the consequences should be distributed among all people. (Sure, there are exceptions if taking too big a risk.)
And personally I feel most medical issues and school should be paid by the state as it would be too unfortunate if an individual should face alone the consequences -- and possibly couldn't afford for an insurance, or is likely not to buy it because has other monetary issues.
Insurance isn't a bet. It's a hedge. The bet is, you're not gonna wreck your car, or burn down your house. The gamble, in your scenario is not getting insurance.
Insurance is precisely a bet that you will burn down your house. It is also a hedge specifically because it is a bet against a desirable outcome. (Either you lose the bet but get the desirable outcome, or you don't get the desirable outcome but at least you win the bet.)
I agree that expected value (EV) is not necessarily a good metric for personal financial decisions. In fact, this applies both to EVs greater than and less than 1.
As you point out, insurance is a good example of a <1 EV; its purpose is to reduce volatility.
Another example: lottery tickets in the occasional case where the EV>1. This is supposed to, for instance, lead a rational economist to buy a lottery ticket (or many lottery tickets!) when the Powerball jackpot hits some particular threshold, say 500 million. However, money isn't linear in terms of value to individuals, and for most people the difference in life impact between winning a billion dollars vs 500 million is not anywhere close to 2x - indeed the two outcomes are more or less effectively identical.
tl;dr: because money is not linear in the value it adds, EV is not a good optimization metric for highly skewed outcomes.
However, if you compute EV as the geometric mean of outcomes, instead of the arithmetic mean, it works again. (This is mathematically equivalent to the log utility you allude to.)
This is the right way to think of repeated bets (rather than in isolation) and Bernoulli's 1734 paper on it is a very readable intro to thinking in terms of the Kelly criterion.
Moreover, insurance is a bet against yourself. When assessing insurance I always go through this exercise. For example, optional auto insurance when renting a car is routinely extremely overpriced. A good way to reason about it is "Would I bet $20/day that I'm going to have an accident in this car for the next 3 days?"
For interest, there's a very common negative expected value bet that almost everyone is required to make: insurance.
We don't consider that gambling, in fact we often tell our parents to buy some when they fly on holiday.
Why? The answer touches on the lottery.
We care about not just the average case, we care about what might happen.
Regarding Kelly criterion, there's a good reason why people don't used exactly the amount it says. If you look at the risk, ie the chance your probability is wrong, there's a chance you are overbetting.