Abandoned Python Projects
At one time or another, we’ve all come across them. I think it’s less common now than it used to be. These days, most open source projects are hosted on Github and easily forkable by anyone who wants to maintain them (compared with a time when many open source projects were hosted on sourceforge and abandoned projects were a dime a dozen).
But even though github has made it easy to fork and maintain a project that has been abandoned, it’s still not very easy to take over distributionof an abandoned project on PyPI.
After the infamous npm left-pad incident, many people justifiably wondered whether something similar could ever befall the python community. Unlike node, which has a handful of different package managers (npm, yarn, bower, etc.), python really only has a single package index — PyPI. And unlike the node package indexes, which in some cases are run by well-funded, for-profit businesses with dozens or hundreds of employees, PyPI is managed by the Python Packaging Authority, a non-profit organization run almost entirely with volunteers and donations. If anything, they python ecosystem is even more in danger.
Fortunately (or unfortunately, depending on how you look at it), until recently, the infrastructure of PyPI lacked the technical tools to remove packages or users from the index. But with the growing number of packages and namespaces, abandoned projects and namespaces will only become more numerous.
All of these concerns lead to the creation of PEP 541 — Package Index Name Retention. It outlines a process by which abandoned packages and namespaces on PyPI can be reclaimed by new maintainers. In fact, the PEP was accepted over a year ago, in early 2018.
At the time PyPI was still handling support requests through a sourceforge issue tracker (I know, that’s so 2006). But hey, at least there’s a process… or so I thought. At some point, they switched to handling issues on github. If you browse through the issues, there are plenty of PEP 541 transfer requests. But as far as I can tell, PyPI has yet to transfer a single project according to the process and requirements outlined in PEP 541.
Meanwhile, I’ve had a PEP 541 transfer request up for over a year.
At one of my previous jobs, I was tasked with evaluating rocketchat as a team chat client. Rocketchat has a fairly decent and well-documented REST api. One of the first things I did was to look for a decent python api for rocketchat. The python api that is linked from their official docs page didn’t seem particularly well-designed. When I first encountered it, it hadn’t been updated along with the REST api. It didn’t support certain endpoints and it didn’t pass along some parameters that had been added to the REST api. It also remapped all the endpoints and arguments to slightly-differently-named pep8 versions, which made it even more confusing to use if you were reading the REST api docs. There were a few other rocketchat api’s on PyPI, but they weren’t any better.
So, I decided to make my own. The names and arguments match as closely to the REST api as is possible within the limitations of python naming. The entire codebase is only a few hundred lines, and a good chunk of that is just comments and docstrings. But when I went to upload it to PyPI, I found that another user had already claimed the rocketchat namespace. That package has a single release, uploaded over 2.5 years ago. The release is broken and doesn't work if you try to install it. That's because it's a broken fork of another rocketchat api (which also isn't very good).
Basically, it’s name squatting. It’s an abandoned package that never worked to begin with. The links on the project are all broken. I’ve tried contacting the supposed owner, but he’s somewhere in Serbia and refuses to respond to any of my emails or github communications. It’s textbook PEP 541 material. And yet, PyPA isn’t really doing anything about it. They’re not even abiding by their own process for handling these situations.
Granted, as I mentioned above, the PyPA is run almost entirely by volunteers, so it’s understandable if PEP 541 transfer requests fall to the wayside in the context of all the other stuff that needs to be done to manage a project as large as PyPI. But it’s still frustrating and disconcerting that the python community hasn’t invested the resources to deal with these issues yet. I’m frankly amazed that PyPI has managed to sustain itself this long as a volunteer organization.
So, if you ever have a project on PyPI and you abandon it, make sure you read your emails so you can hand over maintenance of the project to another developer, so that PyPA never has to be bothered.
