At least some contractors get some of the perks. I've seen kitchen workers, security, maintenance workers eating in the cafes. This article says bus drivers have access to the cafes and gym:

http://www.businessinsider.com/google-bus-drivers-work-long-...

If all the contractors became employees the diversity numbers would improve. But the pay rate correlation to minorities would decline.