You're not wrong. But it should be explicitly noted and let people reduce their arguments to "my priors say an invisible force has a terminal value of consuming drugs being bad".

Then instead of saying how people disagree, we can just note that people have flawed beliefs. Otherwise talking about agreement somehow seems like there's a problem with the correct side, that hey "both are valid" and so on.