The problem is `requirements.txt` doesn't do anything with downstream dependencies. There's nothing like a shinkwrap/lockfile in python. Even if you pin dependencies to exact versions, if you check your project out in a new environment and run pip install -r requirements.txt, you can end up with different, broken downstream dependencies.
If you want to stick with using `pip` over any of the newer tools that build on top of it (Poetry - my favourite, pdm, pipenv, rye, ...) the simplest way I used in the past was to use a `requirements.human.txt` to set my dependencies, then install them in a venv and do `pip freeze > requirements.txt` to lock all of the transitive dependencies.
That's an awareness problem. requirements.txt was invented... a long time ago, I think before the much more sane (but still not perfect) dependencies/lockfile split got popular. requirements.txt tries to be both - and it can be both, just not at the same time.
In short, you want your deployed software to use pip freeze > requirements.txt and libraries to only specify dependencies with minimal version conditions.
I did not know about pip freeze, doh. Thanks will check that out!
Edit: so if I understand it, this is just listing all packages in the current python env and writing them to a file. Hm, requires more discipline than the npm equivalent. But thats a natural consequence of pip defaulting to installing packages globally (vs npm which installs in local node_modules by default). Better but still not awesome IMO
Why would you bluntly assume my comment lacks any foresight? I was simply recommending you a tool that I used, albeit briefly, that solves the exact the same problem for which you are claiming no solution exists.
Nobody is denying that it would be ideal if there is one best solution to every problem in the ecosystem. But at the end of the day all software, including core and third party libs is just code written by people, and it is too much to expect that any person (or a group of them) gets everything right the first time. Change, breaking or otherwise, is inevitable as people learn from their mistakes - its not like the core is guaranteed to never have any breaking changes either.
Just like you can pin the version of libraries, you can pin the versions of your tools too, as long as they are not depending on external services with no versioning. The point of the post is not absolute avoidance of change. It is to opt into a workflow and tooling setup so you can deal with the upstream changes at your own time and convenience.
And BTW, looking at their versioning, poetry hasn't yet had any breaking changes in its 4+ years of existence.
This happens a lot for me.