There's one category where credit scores fail miserably, and that's a very, very small group of people (which I fall into). I have no debt and no open credit lines. I saved up and paid each semester's tuition at the start, in cash for school (this was probably the hardest debt to avoid). I've only made payments on something once (a car) and paid it off after 8 months and I've never had a credit card. It's not that I don't buy things, I just don't buy something until I have the cash on-hand. The fact that my income is double my living costs and I've never had an account go to collections doesn't figure in and I have less (and worse) options available for credit than someone who has been running up credit card bills or other lines of credit and missing payments, defaulting on loans, etc.
Then again, this is an outlier group and the actual problem with the system is that while it's massively efficient and accurate nearly all the time, we're so dependent on them that there is no mechanism to override the system when it's appropriate.
Somewhat off-topic, but if this worries you get a secured card from your favorite bank or credit union. Typically you deposit $500 into a special savings account, they freeze it, and they give you a card with a $500 credit limit in return. It reports like a normal credit card. After 6 months to a year of responsible use of the card (i.e. buy a stick of gum every 3 months, pay after you receive the statement) they will graduate it into a standard credit card and unfreeze your linked savings account.
You can build credit history fairly fast this way. (Starting today with no history, you can have FICO 800 in about two years without paying a penny in interest, if that really matters to you.)
Everything in finance is negotiable, but you've generally got to work for it. You can convince a bank officer to override the computer and grant you credit -- this is a matter of routine. It isn't as easy as filling in a web form at 3 AM in the morning and having a decision in 15 seconds, but for minor credit lines it shouldn't take much more than a branch visit if you have history with the bank. (I know we're all wonderful snowflakes, but a bank with hundreds of thousands of customers may have run into someone who dislikes debt before.)
A slightly bigger problem is with employers using credit score to grade applicants. I'm also pretty close to the same zero-debt blank slate he is (and I'm fixing that), but it can be a problem when people think that a bad credit score makes you a big risk for theft/embezzlement/etc. when you're not.
I think the system works as designed even in your case. By not accepting loans you haven't proved you will pay them back if you get them; you've simply proved that you don't like loans. You might assert that not taking loans demonstrates some sort of fiscal responsibility, but I'd say the hard data collected by the credit rating companies trumps your anecdotal evidence.
Think about it; if someone who hasn't taken loans their whole life suddenly wants a loan, isn't that a bad sign for their fiscal situation? Why shouldn't they have a lower score than someone who has proven they will pay back loans by actually doing it many times?
Then again, this is an outlier group and the actual problem with the system is that while it's massively efficient and accurate nearly all the time, we're so dependent on them that there is no mechanism to override the system when it's appropriate.