Perl 6 uses rational arithmetic by default (or by defining typed variables), which covers the money use case well. And, of course, we have SQL's DECIMAL type in most backends.
And Common Lisp, although it’s not necessarily “default”. (/ 2 3) works as expected, but 1.7 is a float (unfortunately), and not a rational 17/100. Shame, really.
Nothing unfortunate about that. When you work with rationals, it's very easy to grow incredibly large denominators, at which point all arithmetic operations slow to a crawl. Floats are fast and predictable, which is why they're used for most calculations in CL and other languages.