It's a bit like that with every friend that you make. Most people lose contact with firends they made in school. Even family ties seems to be not that strong after all. It's all a matter of context.

That being said, what do you actully expect when spending time with work peers? It's supposed to be just relax after work. If it does not "spark joy" then just don't do it. If it does, then do not expect any other value. Friendship is not about how much you can get from others.