The author of the video says that the Kelly criterion is used in investing, but that's not really the case. Kelly should only be used when the probabilities of winning/losing are known exactly. When they are only estimated, like in investing, the Kelly criterion is still far too risky. Here is a bit more about this problem:
Unfortunately there doesn't yet seem to exist an equivalent to the Kelly formula which would apply to subjective probabilities. I haven't even found an explanation of why uncertainty about probabilities requires betting more conservatively than Kelly would recommend.
> I haven't even found an explanation of why uncertainty about probabilities requires betting more conservatively than Kelly would recommend.
It’s because the risk of ruin increases quickly if you overestimate optimal f, as calculated by Kelly criterion. So OVER-estimating f, dramatically increases your risk of bankruptcy, while UNDER-estimating f only results in not optimizing your returns.
If you’re not certain of your probabilities because (you’re estimating unknowns), you’d prefer having sub-optimal gains over going bankrupt.
I guess something like that is indeed the reason. However, as stated (in terms of "risk of ruin") the argument doesn't quite work. Because even if the probabilities are known with certainty, betting the Kelly optimum would increase the risk of ruin compared to betting less. But Kelly betting would nonetheless be better.
The thing about Kelly is that it assumes risk of ruin is zero because you never bet your entire bankroll. It assumes you can always bet a fraction of a fraction of what you have remaining.
If you don't make any assumptions about the underlying distribution you're associating with the risk, it can be anything, so I'm not surprised you haven't found an explanation. For example, the risk could be dependent on previously values, dependent on unobservable data, completely independent, etc. etc.
If you make some basic assumptions about independence, along with some others that I may not understand so well, you can start to grapple with the idea that you should be decreasing the amount you wager on each round as you're hedging against risk. If you make a mistake in your probability estimation in your favor, then great, you win more, but if you make a mistake against, it bites you and you lose more than you would have thought.
In the extreme, your "deflation factor" is 0, meaning you wager nothing so you're completely hedged against risk.
In terms of the video you provided, he talked about using a sample of 10 to estimate the probability but notice he didn't even mention the variance of such a limited sample. I don't claim to have any deep knowledge of this but Evan Miller's "how not to sort by average rating" post comes to mind in order to get some handle on this particular setup [0]. From the Miller post:
What we want to ask is: Given the ratings I have,
there is a 95% chance that the “real” fraction of
positive ratings is at least what?
Miller advocates for using a lower bound Wilson score confidence interval for a Bernoulli parameter. Again, I'm pretty naive when it comes to this stuff but I suspect this is exactly what would be wanted when trying to apply the Kelly criterion in the example from the video you provided.
The reality is that risk management is a big subject precisely because the risk model can be complex. The Kelly criterion is meant to be a starting point for complex real world examples, not the destination.
I also found a post that has a nice graphic for deflating the Kelly criterion [1], so maybe that might be helpful to you.
> If you make a mistake in your probability estimation in your favor, then great, you win more, but if you make a mistake against, it bites you and you lose more than you would have thought.
There are two mistakes we can make due to inaccurate probability estimation that are not on our favor: Betting too little, and betting too much (relative to the actual Kelly optimum with the true probabilities). I believe betting too much is disproportionately worse than betting too little. But I haven't seen a direct argument for why this would be so.
The Nick Yoder Blog post is interesting, but he basically says: people just happen to be risk averse, and that's the reason we happen to want to bet less than the Kelly optimum.
> Most people assign a negative value to risk. That’s why we pay a premium to insurance companies to haul away excess risk.
But I don't think a psychological disposition is the right explanation here. The original video I linked to suggests that it is actually irrational to bet the estimated Kelly optimum when the probabilities are estimated. No matter whether you are psychologically risk averse or not.
The Kelly Criterion [0] tells what strategy to employ if you're given fair odds on an unfair coin.
For example, if a bet can be made where you get 1:1 payout for a coin that's weighted heads with probability 0.8 and tails with probability 0.2, how much should you bet to maximize wins, under a suitable formulation, if you're allowed to play repeatedly. Betting all your wealth at each round nearly guarantees you'll lose everything the more you play, so it's not as straight forward as one might think.
If you decide to bet a fixed amount of your current wealth at every turn, that's the condition that the Kelly criterion tells you to bet:
K = (bp - q) / b
Where p=1-q is the probability of heads and b is the odds of winning (that is, the coin gives b:1 odds).
There's some SO posts about it [1] and I've also made a short blog post, with minimal explanation, which derives it [2].
Note that this has wider applicability than just "how to bet on a horse race you know is fixed". If you believe the stock market is not zero sum and is increasing by a certain amount, how do you bet part of your fortune to maximize your returns? If you have a cohort of people founding startups with a certain failure rate and return, how much of your funds wealth do you invest in these ventures?
The real world is complex but the Kelly criterion gives a starting point to talk about these things.
I heard somewhere that Warren Buffet uses a factor of 0.9 on top of the Kelly criterion calculation to account for increased risk.
I found this easier to just explain as a factor instead of speaking of percentages. If you win, your money gets bigger by a factor of 1.8. If you lose, your money gets smaller by a factor of 2 or multiplied by a factor of 1/2. So you find your end value by just multiplying all these factors up.
Obviously if the number of winnings and losings is the same, you’ll be multiplying by 1.8 as often as dividing by 2, so on average your money will get smaller by like 0.9 each round. Not counter-intuitive if you think in terms of factors.
There's a related idea, centered on the St. Petersburg paradox. It's the idea that the median outcome is sometimes a much better predictor of human behavior than the mean outcome: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3811154/
The St Petersburg lottery is a good counterexample against basing decisions on (arithmetic) mean outcome; AKA the expected value.
In particular, the arithmetic mean is useful in situations where the source of value is irrelevant, we only care about the total. For example, if I can earn $X/hour doing job 1 and $Y/hour doing job 2, then (assuming they're equally difficult, fulfilling, etc.) my total value should be the arithmetic mean of spending the fraction t of my time doing job 1 versus job 2:
t * $X/hour + (1 - t) * $Y/hour
This makes sense, since the money I get is the sum of the money from each job. In particular if I "waste" some time on a low-paid job, I can "make it up" by spending some time on a well-paid job, resulting in a total value somewhere in-between. To maximise my value, I go "all in" on whichever job pays most.
Games of chance are not like this, since the source of value is highly relevant. We'll only receive one of the possible payoffs: if it's a low payout, we can't "make it up" using some of a high payoff that didn't happen. Hence we don't directly care about the "total" of all the payouts, since we can't aggregate winnings from alternate timelines. Instead, we should maximise the geometric mean of the payouts, which makes each individual outcome the least-bad. The result is the Kelly Criterion.
The St Petersburg lottery makes this clear: I don't care how exponentially-huge one of the payoffs might have grown, if the payoff I actually receive is $1!
The same logic works across other sources of value, e.g. GDP-per-capita is an arithmetic mean; a government can maximise it by going all-in on the sector with the highest return (e.g. the US could divert its military budget, food subsidies, etc. into the tech sector); or the region with the highest return (e.g. the UK could invest everything in London); or the age demographic that's most productive; etc. However, as an individual human I only have one age; I work in one sector; I live in one place; etc. so I'd prefer a government that spreads its investments more like Kelly betting.
Scott Garrabrant wrote a nice series on Less Wrong which explains/derives the Kelly Criterion, and relates it to other areas (Nash bargaining, Thompson sampling, Bayesian updating), and how they all follow the same underlying logic (optimising the geometric mean) https://www.lesswrong.com/s/4hmf7rdfuXDJkxhfg
I recommend Fortune’s Formula, a book about the Kelly criterion by William Poundstone.
The book's blurb claims it "... traces how the Kelly formula sparked controversy even as it made fortunes at race tracks, casinos, and trading desks. It reveals the dark side of this alluring scheme, which is founded on exploiting an insider’s edge."
The book is light on math and strong on stories about and interviews with people using the Kelly criterion.
Here's a fun game where you have 10 minutes to flip an unfair coin. In my experience I get the best results when I just move really fast and don't worry about matching Kelly exactly. https://elmwealth.com/coin-flip/
https://youtube.com/watch?v=-9JM9suCIHs
Unfortunately there doesn't yet seem to exist an equivalent to the Kelly formula which would apply to subjective probabilities. I haven't even found an explanation of why uncertainty about probabilities requires betting more conservatively than Kelly would recommend.