Prefer "learning" rather than "remembering." I prefer learning by doing and teaching others which coincide with the best ways to learn something and be held accountable. I don't remember where I first learned about learning but this pyramid gives a quick breakdown of which methods work best for retention [1].
It is not a dichotomy and do learning without remembering does not exist unless we have such different definitions of learning that your definition does not imply any structural change on the brain.
[1] The Learning Pyramid: https://stephenslighthouse.com/2010/02/26/the-learning-pyram...