> Isn't rhe problem wirh dense embedding is memory usage?

Not necessarily, given that a dense embedding can encode the equivalent of many many words or even higher order concepts not easily expressed in word space.