I always thought that college was not to increase your typical office worker's productivity by X% but instead to be a long term bet on massive gains for society through academic research.

If that's the true, we should make colleges more intellectually elitist and put most of the funding on places that provide that better worker's productivity, or simply cut funding if there is no better option.

We also should make it easier for those academics to bring those massive gains, because academia isn't organized for that.

I still think this is not the real goal, but maybe it was at some time. (Is there more than one goal? The system appears to not be optimized for any reasonable goal.)

That's probably not the (only) way to do it. There are two interrelated but distinct types of breakthroughs that happen in research: the kind that occurs well within expectation, the next step in a well-understood plan; and the kind that is serendipitous, usually by way of uncommon or novel interpersonal or intellectual communication. For the latter, you need... diversity. Expertise, too, of course, and some amount of common ground. However, the most interesting and creativity-enabling interactions require people who aren't all aiming at the same thing before they initially interact.

On top of that, it can be difficult to separate the people who are actually intellectual elites and the people who are good self-marketers.

I kind of think we should just be committing a lot more funding to those interested, who meet some reasonable, objective level of expertise.