I imagine these stats have a lot more to do with the roughness of the water than the temperature. Rough water can be extremely difficult and stressful to stay up in, especially if you're not used to it.
Also, if you get tossed off a boat in the North Sea, chances are that the water is not very calm.
It can depend on the individual, too. There's an interesting book called Swimming to Antarctica, which is a memoir by Lynne Cox, an incredible long-distance swimmer who discovered she had a talent for cold water and who (after extensive preparation) swam five miles in sub-40º water.
One of the exceptions shown was a guy who was asleep below deck when his fishing boat sank. Iirc he treaded water for a while as all crew mates succumbed one by one. He then swam to shore (miles i think) then walked several more miles to a house.
I think he collapsed on the doorstep (again having "made it") but was okay in the end.
He was not a slim guy, not sure if that helped.. but it was one of the dont give up examples. So outcomes do vary wildly.
Obviously take those statistics with a grain of salt.
According to some stats you can survive one minute in icy water.
Yet many people enjoy swimming in the ice, including some who can stay in the icy water for 15 minutes or more just for kicks.