Regarding your points

  * First, state-of-the-art was accessible and well-documented with designs reflected in academic literature.
  * Second, implementations were relatively simple and straightforward.
  * Neither of these is true today.

Again the situation is more nuanced than this. Before the web systems were expensive, resources were scarce and connections to research were critical. Now anyone with time and a laptop can watch youtube lectures, read recent research, clone a repo and join the party. Different things are hard now. Such is the way of competition. Hardware and economic trends move the state-of-the-art quickly. Ideas offering competitive advantage are rarely documented to the degree one may like. But they do find their way to entrepreneurial practitioners and students. Their time, energy, attention and motivation builds new and better systems and we should welcome their efforts when they advance open-source projects.

> I'm not sure I'd be able to bootstrap the necessary expertise to build database engines today like I did back then.

Don't sell yourself short :-)