The Congress shall have power...To promote the progress of science and useful arts, by securing for limited times to authors and inventors the exclusive right to their respective writings and discoveries;
I think we should recognize there can be a strong misalignment between what’s written in a text and its actual purpose or how it will be enforced.
For instance the patriot act surely was filled with feel good vocabulary about protecting the public good, net neutrality’s repel “to increase investment”, and I’m sure the coming up “save the children” act that will aim at giving better education and critical thinking to younger generations.
There doesn't seem to have been much of a lobbying process. It was accepted with little debate, and other options to promote progress were rejected. Copyright was a well-established practice inherited from British common law, and viewed by Madison to be clearly in the public interest:
"The utility of this power will scarcely be questioned. The copyright of authors has been solemnly adjudged in Great Britain, to be a right of common law. The right to useful inventions seems with equal reason to belong to the inventors. The public good fully coincides in both cases with the claims of individuals. The States cannot separately make effectual provisions for either of the cases, and most of them have anticipated the decision of this point, by laws passed at the instance of Congress." Federalist 43
At least with regards to copyright, it is pretty well known that Noah Webster was lobbying powerfully for it [1].
The quote you have from Madison no doubt reflects his real opnion, and well the established view in Britain. But the reason he needed to write it in his letters was that he had to argue against the opposite view among other founders.
You can guess which side Webster supported. And this is the kind of thing I mean about it being a "well-known public justification".
The Federalist Papers were campaign propaganda for the ratification of the Constitution; a comment in the Federalist Papers is not a basis for rejecting the possibility that the position put forward in that comment is a public rationalization rather than the real motivation, any moreso than would be the case for any argument put forward in any other work of political campaign propaganda.
Accepting the terms of copyright protection then is very different than from now. "Limited term" is no longer observed for anything of value to a large content publisher.
https://www.law.cornell.edu/constitution/articlei#section8
Yes, patents existed before the US Constitution, but the reason the practice was formalized in the US was to promote progress.