> technology so advanced it might as well be magic

All "optimistic" sci-fi is reliant on a utopian fantasy in some form, I wouldn't be looking to sci-fi authors to actually have a workable recipe for how to achieve any given desired societal outcome..