I think you've missed his point here. What he was saying is that the purpose of college is only partly to give an education. The main thing college gets you is credentials and socialization. If what you want is just a good, broad education and you don't care about those other things, then the library is provides that to you.

On the other hand, the collections (books, journals, etc.) of college libraries are typically vastly superior to those of public libraries.

I really miss the access now that I'm no longer in college.

In any public library (in the US, anyway), you can check out books from every library that is part of the inter-library loan service. This include all (I think) other public libraries as well as a healthy number of college libraries. You're not even limited to books!

Have a chat with your local librarian. You might be pleasantly surprised at how deep that resource goes.

I'm aware of that. Nonetheless, I can say definitively that the South Central Library System of Wisconsin (which includes Madison) does not include the University of Wisconsin libraries, and the holdings of the latter are vastly superior, at least in terms of academic-related material. I've not been pleasantly surprised.