I have volume and display brightness buttons, also a mute button and play/pause. They are permanent and don’t switch with the application. Pretty much like it was before the Touchbar....

I tried to use the Touchbar for a while without customization but I never could get the hang of it. It just seems useless.

Is that different from System Preferences > Keyboard > Touch Bar shows Expanded Control Strip?

You have more options. But it has been a while since I set this up so I don’t remember exactly.