I think perhaps you are reading a meaning into my comments that is not intended. At no point have I argued, nor would I ever argue, for some waterfall-based approach and magical everything-fully-known-up-front documentation.

My argument is that whatever stage you’re at in your development process, if you don’t have clear requirements, you don’t know what you’re trying to build and you have no way to define whether your software is working or not. The comparison from the Agile Manifesto makes little sense, because having comprehensive documentation of your requirements and acceptance criteria is a prerequisite for even being able to define the working software that the Manifesto says is more valuable.

Of course you might do all kinds of prototypes and proofs-of-concept to help you explore the problem space and clarify the requirements in the early stages of your project, and maybe again later if there are significant changes. But this work isn’t about building working software, because you don’t know what that means yet; it’s about doing experiments to help you find out.

yeah but if your "comprehensive spec" is neither comprehensive or even accurate, how is your software working any better? If "working" means "what the spec says" instead of "what the customer/stakeholder wanted" then your "working" software isn't working.

Providing the customer with a working prototype to criticise often (always) allows them to refine their vision of what they wanted and helps them communicate what they need. It's always easier to criticise something than create - saying "the interaction around X doesn't feel right, can we make it cornflower blue?" is a lot easier than "the X interaction must be three shades of grey off cornflower blue" in a spec document.

That's been my experience, anyway. Insisting on a comprehensive spec at the start has always been more effort than its worth, because no-one knows what they actually want at the start.

This has become a frustrating discussion, because it seems like several people read the words “comprehensive spec” and inferred some sort of waterfall process with everything implausibly known up-front. That is not at all what I’m arguing for. In fact, I’m not sure I’ve ever seen anyone argue for it in a professional context. It just seems to be the hypothetical bogeyman in these discussions.

In the case you’re talking about, creating a prototype and then soliciting feedback from the client might be an excellent way to go. More generally, building a project in stages with regular interaction with and feedback from the customer or other stakeholders is often helpful. My point is that you still need to decide what you’re putting in that prototype or any of those earlier stages, because ultimately some developer has to write that code and management need to tell them what to create.

So you might well have a spec, in whatever form, that has initially has gaps or ambiguities relative to the final product you’ll want at the end of the whole process. That’s fine, and as you say, it is very common. But it should still reflect everything you know you want at the current time.

If and when you gain new information, such as feedback from your client after showing them a prototype, you can update your requirements to fill in the gaps or clarify the ambiguities in light of the additional knowledge you now have. You still have a clear record of what is needed, taking into account all the actual decisions that have been made and all the actual feedback you’ve received, and that is (to the best of your knowledge at that time) what defines whether or not you currently have working software.

Put another way, if you don’t maintain a good record of that information, you’re just building your software based on hearsay. Some of that information about what the customer/stakeholder actually wants will inevitably be lost or distorted with the passage of time and perhaps the changing make-up of your team. And then you definitely don’t have working software, whether or not you realise it.