Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

How big can my world be in meters if I require at least 0.01mm precision and I model things in millimeters? Does that number change if I model in meters?


Because of how floating point works, it doesn't matter. The difference between modelling in meters or millimeters disappears in the exponents. What matters for precision is the mantissa, which is 23 bits in single precision floats. So you get ~16 million of your smallest unit before you're losing precision, counting the implied leading 1. You can double that to ~32 million by using the sign bit.

Edit: this translates to 320m if you want 0.01mm precision.


About 0.008 km^3, each coordinate between -100 and +100 meters.

The number's the roughly the same even if you'll model in km or imperial units.


Yeah, you're right. Brain fart. I'm mis-remembering production stories instead of thinking clearly about floats, despite having just linked the precision chart. :P




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: