Backprop was popularized in 1986. Transformers came out in 2017.

It could be >10-20 years before the next breakthrough comes. Given all the progress so far, these things still don't seem to be learning any logic.