Depends what you need to host. If you just want email & a blog, gandi.net will providing hosting for those as part of their domain registration (~$15 per year, depending on which TLD you want).
They also include minimal web hosting with registration too, but last I looked their (free) offering was so bad it didn't really count as a feature.
I imagine some other domain registrars have similar options.
I got a domain through gandi and decided to try my email there for a change, it's been pretty reliable and solid, the only time I needed support it was quick. I don't however do a lot of email so my experience might be much different to a power email user.
They also include minimal web hosting with registration too, but last I looked their (free) offering was so bad it didn't really count as a feature.
I imagine some other domain registrars have similar options.