There needs to be a balance. You should choose an architecture which will accommodate predictable future needs without too much refactoring. A suite of unit tests can't help you much if you need to unmangle a bunch of severe abstraction violations between what you now need to be well-encapsulated components.
Actual red-green-refactor TDD results in the simplest possible code to solve a problem, IME. Over- or under-abstraction tends to be pretty well instantly recognizable when working with a well written test suite, and can then be easily and safely refactored away.
