Somehow pipenv got endorsed as an official tool by PyPA far before it was ready for production. The documentation for pipenv and Python packaging didn’t make it clear early on pipenv was for applications, not libraries.
Also the pipenv maintainers were very rude about closing bugs. This has lead to a lot of reasonable criticism about the tool and how it’s maintained.
Despite marketing the tool heavily, Reitz is taking all of this personally and is blaming others or his mental issues.
Now, Reitz has posted this 3 times on /r/python in the past 2 days, and hosted a hangout while also saying he doesn’t have time for criticism.
And he and his PyPA buddies are all posting in all the threads and making passive-aggressive tweets.
To top all this off Reitz has now made and mods /r/positivepython in an attempt to imply the original is somehow not “positive”. And in my opinion to exert more power over the python community.
Before all this I really respected him, but this childish out burst will really make me think about how well design or endorsed his work is in the future.
You write "The documentation for pipenv and Python packaging didn’t make it clear early on pipenv was for applications, not libraries."
The linked-to essay says: "There is a misnomer that Pipenv is not useful for libraries. That is false. Try using $ pipenv install -e . and see what it does when you lock. It augments setup.py. I, as a library developer, find that extremely useful."
I can't help but conclude that the documentation doesn't make that clear because the author doesn't think it's true.
I confess that I don't understand the issues. Then again, I only last month migrated my tools from 'distutils' to 'setuptools', so I know I'm way behind the times.
I have just now read https://docs.pipenv.org/advanced/#pipfile-vs-setuppy on the topic and am still confused by the terminology. Eg, "Applications are different in that they usually are not depended on by other projects."
I am used to writing command-line applications which are used as dependencies for other projects.
But that is neither here nor there - no need to answer. Thanks for pointing out the nuance I did not recognize.
> Somehow pipenv got endorsed as an official tool by PyPA far before it was ready for production.
Pretty much nails it for me. Being up to date with new technologies and tools is important, and once it became officially endorsed I (and many others) had to run with it.
It was about 3 months I tried it last time, but at that time it was still slow, it ignored ~/.pip/pip.conf and also had no "proxy" CLI option like pip does.
I hope it have changed now but early on it also had some strange preference to environment variables rather than CLI options, which I find odd for an application aimed for command line.
I voiced my concerns about these issues, because why not? If I as a developer release something and intend to continue maintain it, I have a responsibility towards the users that depend on my creation. Old-fashioned stance perhaps...
I did a quick browse through the threads in question and suggest others do the same. Yes, there is negativity, but the actual debate is more complex than simply one grumpy side slamming a poor developer.
The most relevant comment I found was by /u/bixmix:
> It's getting much worse - not better. The internal churn within pip also is not community driven as much as it is developer driven. It's like we didn't learn from the debacle of python3 vs 2.
I'm not a Pythonist so I had no idea there was such a huge debate over how to manage deps. At various points the past few years I've googled "bundler for python" and come up with different projects. I remember trying pipenv early on (didn't work with conda Python install). Then I revisited it months later--the project was much more developed--and it worked great. FWIW I like pipenv, it solves my largest gripe with Python development.
If you have as part of your community philosophy "there should be one—and preferably only one—obvious way to do it," it seems like you're going get a lot of stubborn tug-of-war situations (see 2 vs 3) where each side thinks their way is the superior way and there is less room to compromise for the sake of keeping the community together. That said, this pattern of debate is pretty universal across communities.
Ruby--even with it's MINASWAN motto--had it's own RVM vs rebenv drama back in 2011. Same sort of developer/community vs developer/community drama. Nowadays I think rbenv has a larger following.
As a fellow rubyist in python land, I’m also a fan of Pipenv because it uses a model I understand very well - that of Bundler (and similar to that of Cargo and Yarn). I do have some technical issues with it, like the strange hiding of the virtualenv in a random-like location instead of .venv by default, but overall it’s a large improvement over requirements.txt/setup.py/frozen_requirements.txt and even the new toml thing.
Maybe there is some inside baseball going on here when it comes to python foundation etc, but as an outsider looking in the vitriol seems totally unnecessary. I wouldn’t develop Python software w/o Pipenv.
Bundler also had some drama early on in its lifecycle as well. At least one Rubygems developer actively disliked it for quite a while. But, after a concerted effort for a couple years, Bundler managed to hammer out the issues and come out with something that could meet a wide range of workflows and really set the standard for what dynamic language package managers should be capable of. I think this is a real testament to Yehuda Katz's strength as an open source community leader.
I don't use python so I'm not involved in the controversy either way, but it's pretty encouraging to me that this person is open about the limitations of his humanity and its effects on his (apparently very popular) project. He acknowledges it and has taken steps to limit its impact on the project.
We'll probably never know many projects have withered on the vine because of similar issues. Kudos to him for his open letter and willingness to deal with the negativity in a meaningful way. (It always kills me when people attack others for providing free fucking software.)
People are rightly turned off that the Python Software Foundation (which Kenneth is a core member of) declared PipEnv the official package manager[0]. It reeks of favoritism, insider politics and the cult of personality surrounding the tool's author. It would have been better if PipEnv proved itself as a better tool in the open marketplace of ideas.
Python the community is full of dogma. That is why I don't like the language and I stay away from the community. Their own ideas come before objective truth to them.
I saw the first of the posts he mentions, and yes, I was very surprised to see the negativity. There is literally no reason to be negative, if you don't like the tool, use another. People seem to be upset that Pipenv is the recommendation now, but it's better than pip, and it should be the recommendation until something better comes along.
I'm more upset about the whole pyproject.toml thing that means I have to migrate everything again, from Pipfile this time, even though I have barely finished migrating to it!
pyproject.toml isn't another way to specify dependencies for Python code.
The long version of why it exists:
Historically, a Python package has been defined by the existence of the setup.py file. Executing this file with various command-line arguments is how packaging tasks are carried out: 'setup.py install' installs the package, 'setup.py sdist' builds a source distribution (.tar.gz) suitable for uploading to a package repository, etc., etc.
But this workflow has dependencies! Most packages will only depend on setuptools (which supplies the setup() function) and wheel (which supports building binary packages), so the packaging tooling assumes the presence of setuptools and wheel and you never have to specify them as explicit dependencies (in the old old days, setuptools used a drive-by installer file bundled into the package so people who didn't have setuptools could still install setuptools-based packages; nowadays, pip supplies it automatically).
But what if you have a workflow that requires something else? What if you use an alternative package manager, or package build tool, or really anything that isn't setuptools or wheel? How do you specify the toolchain you need to work with your package?
That is the thing pyproject.toml solves. Since it's not an executable Python file, it can't get stuck in a chicken-and-egg problem ("execute setup.py to find out what you need to execute setup.py"), and it can specify the necessary build system for a package in a standardized way.
Yes, I understand the purpose, but the result is that it replaces any previous way of specifying dependencies. It's great that it exists, but it's frustrating that it was developed right when the new thing for specifying dependencies (Pipfile) was starting to become popular.
If you think it's another "thing for specifying dependencies" in the category of the pipenv Pipfile, you have not understood the purpose of pyproject.toml.
If you think that's what I think, you haven't understood my comment. Again, I'm saying that I don't want to migrate to yet another format so soon, even if it does much more than just specify dependencies.
Suppose you are developing something which needs to make HTTP requests. So you have a dependency on the requests library.
pyproject.toml is not the place where you declare that dependency. This is why I am pushing back on you, because your comments all very very very strongly imply that you think pyproject.toml is a place where you would declare that dependency.
pyproject.toml is where you'd declare that you're using an alternate build/packaging toolchain. You would then use whatever mechanism that toolchain provides for specifying your dependencies (in the case of the default setuptools build/packaging toolchain: install_requires in setup.py).
Okay, I see what you mean now. I was confused because Poetry declares dependencies in the pyproject.toml, but looking at the actual file, it does it in a Poetry-specific section, so it's not compatible with any of the other tools:
Alternative packaging toolchains can use pyproject.toml that way; it's an extensible file.
But you've caught on to the key thing, which is that neither they, nor the default setuptools-based toolchain, have to do that. The pyproject.toml file exists to solve the chicken-and-egg problem of specifying that you use an alternative toolchain, when figuring that out might otherwise be dependent on your preferred toolchain already being present and configured.
I used to buy into the premise that anonymity + Internet = lots of people acting badly. Social media over the last decade has taught me that anonymity is absolutely not a meaningful requirement (I believe the masses of users, over that time, have adjusted to being comfortable with publicly being jerks without much concern). A lot of people feel no need to have a good reason to be jerks, it's their character, it's who they are all the way down. Some people are just rotten and there isn't much more than that to it (as it pertains to dealing with them as shit throwing types on the Internet). I feel like the author is suffering from not understanding or not accepting that this is the case, that in many or most instances, a person doesn't have a good reason for their terrible behavior online.
The reason for the clichéd build you up, tear you down, repetition process throughout all of human history, is that a vast number of people will be rage monsters, petty, filled with envy and jealousy. The types that are prone to that ill behavior, will come pouring out of the woodword once you've acquired any consequential amount of success/money/fame/whatnot. These are not good people, they are not capable of reasoned discourse, their opinions do not matter, their hostile negativity should be ignored and disregarded accordingly (they're also, fortunately, extraordinarily easy to spot and separate out from the more reasoned criticism; their entirely unnecessary over-the-top negative behavior provides for easy filtering).
I believe they have different use cases. Pyproject.toml is more akin to setup.py (and publishing libraries) where Pipfile is to replace requirements (and building apps). All the people arguing over the “blessedness” seem to be ignoring this.
I think pyproject.toml is meant to be a superset, which means it would replace the Pipfile as well as all the other files. It's a welcome change, but I hate that we went through the short-lived Pipfile intermediate step.
Gah, why would anyone release software into the public domain if this is the type of thanks they get.
I mean, I have no idea if he's right, wrong, or sideways here, but what is clear is that he's released open source software and people are hating on him for it. It's free! You don't have to use it, do you? What gives anyone the right to throw hate towards an open source author?
Whenever you feel that way, tell yourself that 5% of humanity is fundamentally somewhere between horrible and really horrible, capable of doing bad things to their fellow humans without a moment's remorse. Do that while not forgetting that the other 95% isn't like that. You're mostly (not exclusively) going to hear from that 5% when it comes to criticism or hostile feedback, they're loud and they're mean, they often lust to harm others (the outward projection of their own internal issues). You'll far more rarely hear from the kinder 95%; I'd suspect that effect is in play here, as it so often is.
My wife is bipolar type 2 and many people on my father's side of my family are bipolar tyle 1 so I partially understand your struggles.
Few people truely understand what bipolarity does to a person and their family and I just wanted you to know as someone who does:
Keep doing what you have got to do to make it through the day!
Python as a language is incredibly productive as someone who has used it for almost 20 years. The secret to being happy and productive using the language is to completely ignore the noise though: PyCon, blogs, reddit, "blessed frameworks", BDFL, etc.
This will make you as happy as reading the New York times every 30 minutes to find out what Trump is up to, or being on Facebook or Instagram and making yourself envious of other people's lives.
Literally NOTHING matters in Python other than things on a yearly cycle. Don't worry be happy, and just unsubscribe to all things Python that happen at the Micro level and your appreciation of the language will increase.
I guess I’ll never know what drives people to talk about the creators and maintainers of the tools they use in such a negative way. Especially given how little they actually know about them! It also strikes me that the moderators of r/python could be doing more to curb the personal attacks.
WRT to not being able to use pipenv for multiple python versions in a project - i.e. a library, it's been a couple of years since I've writing some substantial python libraries, but do people not use tox anymore? Why can't people stick to the usual virtualenv + pip + tox routine?
My two cents is that conda and conda-forge is the “right” way forward, and attention focused on pipenv is not suboptimal because of pipenv (though you can make engineering arguments against it— after all, it is still very young and has not yet matured the way other packaging tools have).
Rather it’s bad because it’s yet another set of spec files and environment tools to learn, potentially further fracturing the community and taking attention away from other approaches.
To be fair, it’s not really a problem with pipenv, but rather with any “official” community endorsement of pipenv. Saying it’s a third party, optional tool and you’re free to have at it if it’s good for your project is fine. But the minute there’s any degree of officialness to it, it just creates yet another fissure in the anarchy of Python packaging.
My reaction is more malaise. Guess I’ll have to learn about this new packaging gadget too, to be current and signal how with-it I am. But in any case I have a choice, I’ll still just use conda and Docker for application & environment management, supplemented by pip and virtualenv when appropriate.
Current conda dev lead here. Just want to reference an issue we'll be tackling ideally on the timeline of the next 12 months. https://github.com/conda/conda/issues/7248 Conda is not without its problems in the workflow that pipenv addresses. We know we have a deficiency here, and addressing that deficiency is one of our enumerated highest priorities.
Thanks for your work! I agree these are valuable design features to add and to get right, and also agree that pipenv offers tools for this workflow.
From my perspective, I’ve tended to prefer Docker for this. This allows me to version control environment components like environment variables in the form of the Dockerfile. I usually have a Makefile for my project which then uses multistage builds or other techniques to automatically detect when environment.yml (or package changes, like to setup.py) take place, requiring a rebuild of the container, so that downstream things like running tests, dropping into an interactive debugger in a full environment (with the right python environment, environment variables, etc.), all just become simple Make directives.
It doesn’t cover everything. Notably, for situations when there are many target environments, like testing a library I intend to distribute cross-platform, across Python 2/3, etc., I might still use a tool like appveyor rather than suffering the multiplicity of a bunch of Dockerfiles and Make commands to target multiple environments.
But for most applications I develop with an intention of targeting a small set of production or dev environments, it turns out to be better IMO to separate the Python environment & package definition entirely from the larger runtime environment (ENV vars, etc.), and use Docker for that part, with Make as the bridge.
Combined with making a library choose sensible defaults for ENV vars, copiously documenting them, and providing examples, this has worked well enough that I do not need or want to mix the external environment configuration with the app’s config files or package spec, in a “separation of concerns” sense.
Your mileage may vary, but this pattern has really worked well for me, and makes me skeptical of solutions that approach combining environment configuration with packaging as a first-class feature, rather than as a special case.
There is such an air of entitlement with open source these days.
I guess more and more developers don't remember when all the coolest shit WASN'T free. You had to pay for compilers, editors, manuals, etc.
If someone writes a piece of software that you invest into your pipeline, understand that without some sort of contractual agreement, that person isn't OBLIGATED to maintain that code, much less give a shit about your feedback.
Is Ken smug?
Yes.
Is this post full of insufferable little whiny points?
Yes.
But is Ken going to have to do what the python community wants wrt pipenv?
Hell no.
Granted, a lot of this conflict could've been done differently and with better tact, but the gritty fact of the situation is that no matter what happens, it's not his fault for giving away the work. Even if the whole world depended on a module, it's still a voluntary action.
I am not familiar with this particular situation, but while baseless entitlement is bad, criticism is good, and when you publish something with the intention that others use it---and reading his post it seems to me that what he does it not open sourcing a tool he made for himself, he wants people to use pipenv---you are no more the sole owner of it. More so if other people put effort (i.e. patches and bug reports and proposals) in to it. The OP says:
> My response to this comment is — first off, my fame, while certainly categorized under “cult of personality” is not necessarily accidental. It’s called marketing. I worked very hard at becoming well known within the Python community, and toiled away at it for years.
after quoting
>> If only these people didn't advertise themselves as the next best thing to sliced bread, collecting undeserved fame, jobs, influence, and as consequence harming very large number of people and technology in general, I wouldn't mind them doing whatever projects they like.
(w/o attribution, unfortunately), which shows that, if pipenv is a sub-par but popular solution (I did not ever use it), this latter quote may be a completely valid criticism.
I am diverting from my point: if Ken wants and encourages the python community to use pipenv, then he might have to obey that community, and he better does. But if he's releasing pipenv just to give back to the community and share his solution to his problems with others, then you're right. After all, if the participants have no regards for each other, that's a bunch of selfish people, not a community.
I don’t think there’s any such thing as a ‘python community’ and the vast majority of people that use python don’t know about any of this political bullshit.
It’s not just open source though, because of his position Reitz had pipenv blessed by Heroku as the default and endorsed way to manage python packages as early as March.
Reitz is (was?) a developer at Heroku and pushed it as the solution for paying customers.
You don't really think that, you're just being adversarial here.
Heroku is not even a part of the equation of whether or not he's responsible for oss that he authored. No matter where he worked or any of that.
It's not his problem that people are using his software and have committed themselves to have expectations over something they have no right to try and control.
I guess more and more developers don't remember when all the coolest shit WASN'T free. You had to pay for compilers, editors, manuals, etc.
I've been a Microsoft developer my entire career of 20+ years and I remember starting out in the mid to late 90s after graduating from college and all of the tools you needed to learn you had to pay for. But because of both open source tools and the web, the need to develop "Windows Software" has declined precipitously.
Microsoft has had to make a lot of their tooling free, open source and cross platform (Visual Studio Code and .Net Core) and even thier proprietary stuff has had to get much cheaper for small developers. Thier online hosted version of TFS - Visual Studio Team Services - interops with everything.
I don't see myself leaving .Net for back end development. But, I don't have to pay the Windows tax anymore in terms of money or hardware resource requirements.
In fact someone (you? don't care) already shared your blog to r/python instead of posting your entry as a text post. so why are you sharing it here? r/Python is already having dialog on r/Python based on the blog post where a guy talks to r/Python.
this is really bad Rediquette, dude. not upvoting your transparent attempt at generating SEO points
tl;dr: Ken is upset and amazed that disrupting the workflows and toolchains of entrenched Python developers is causing a backlash, and that negative things are being said about him.
The post is self-aggrandising and he has the audacity to think he knows what is best for everybody. There is also the real chance this tool will replace other tried and true tools and workflows with a more complicated one. It smacks of systemd vs upstart vs sysv
> Ken is upset and amazed that disrupting the workflows and toolchains of entrenched Python developers is causing a backlash
I'm not a huge fan, but how is he disrupting the workflows and toolchains? He wrote a tool, if you want to use it, that's cool, if not, that's cool too. It's not like he came to your house and forced you to use Pipenv instead of what you were already using.
If you want something to be mad at, it's the larger issue: We just started moving to Pipfile for dependencies and now we're switching again to some new project.toml file? The tool we use over this file is pretty much irrelevant, since all that matters is this file, and you can use any one of a myriad of tools to manage it and install your dependencies with.
If you don't like Pipenv, use something else that can install stuff from a Pipfile, but why are we making yet another standard to replace all previous ones (xkcd here)?
TL;DR I'm famous and contributed to the Python community, I'm depressed (it's real! Check this references) because people don't like me forcing them to use my tool. these people must be awful.
'I consider myself to have been mildly underpaid, considering what I was responsible for — the entire deployment experience (including uptime) of Python apps on Heroku, one of the largest deployment platforms in the world.'
So, you're saying you don't have enough career experience that this is surprising to you?
Megacorps are outsourcing their core processing systems to 3rd level subcontractors, and these are the systems that literally run the economy itself, and have been doing so for nearly 2 decades now if not longer.
Is this a just or fair situation? no (in my view) But hardly new or surprising.. and certainly not relevant to the point you're trying to make.
Also the pipenv maintainers were very rude about closing bugs. This has lead to a lot of reasonable criticism about the tool and how it’s maintained.
Despite marketing the tool heavily, Reitz is taking all of this personally and is blaming others or his mental issues.
Now, Reitz has posted this 3 times on /r/python in the past 2 days, and hosted a hangout while also saying he doesn’t have time for criticism.
And he and his PyPA buddies are all posting in all the threads and making passive-aggressive tweets.
To top all this off Reitz has now made and mods /r/positivepython in an attempt to imply the original is somehow not “positive”. And in my opinion to exert more power over the python community.
Before all this I really respected him, but this childish out burst will really make me think about how well design or endorsed his work is in the future.