Things I've learned serving on the board of the Python Software Foundation

rednafi | 170 points

The elephant in the room here is the ideological capture of the PSF via its CoC working group. It seems, at best, the power of an unaccountable body to arbitrarily enforce membership conditions (and the like), has not been properly thought through (though, of course, attempts were made). The absence of due process and accountability to the community means the members of this group have arbitrary power to shape the PSF+community however they wish.

They have thus far, explicitly equivocated community criticism of the PSF with "attacking" it; equivocated mentioning the difficulties some communities face with defending these difficulties; and equivocated mentioning the identities of victims of this enforcement with the converse bigotry that people of other identities ought be harmed.

Thus at this point there is no open question about whether the CoC group is enforcing the CoC or using abusing their juridical power to humiliate, defame and exclude members of the community which are critical of the PSF.

This is so repulsive to any reasonable person, the behaviour here is so public and so clearly abusive. Left to their own devices, a PSF operating under this capture will alienate significant numbers of the invested members of the python community -- who are aware-of, and concerned-by, such actions.

I think those of us merely observing this, with some stake in python as a language and community, ought make our repulsion clear. Since this is having, and will have, a serious deleterious impact on those most inclined to participate and invest time in projects of this kind.

mjburgess | 9 months ago

It is sad that open source is all about foundations and positions now, not about software development.

Periodically you see "I am awesome and served on the Steering Council" posts on Stack Overflow.

Some of it from people who have never been on the bug tracker, who don't know or ignore the significant social and abuse of power issues in Python.

The PSF marketing completely ignores reality: Free developers have been crowded out, chased away, humiliated and libeled and replaced by a clique of true believers.

I would not advise anyone to spend significant resources to donate free work to Python. I'd also warn Microsoft employees who are potentially lured in to work on a JIT that they are up against ruthless politicians and should probably better work on C# and F# for their careers.

lrhag | 9 months ago

Interesting, I always wondered how pip was funded. It works too quickly and seamlessly for a free service, interesting to see a company just sort of bankrolls it. I suppose if Fastly will stop bankrolling it then some of the big tech like Microsoft or Amazon should. It is the world’s most popular language now after all, and a lot of ML codebases are exclusively in Python.

sashank_1509 | 9 months ago

Foundations seem critical for the health and longevity of ecosystems. It is interesting to compare the various foundations in other ecosystems, their tax/entity status, and who comprises the board.

Many boards are structured to purely benefit the company “sponsoring” and this is wrong. I’ve always looked at PSF as a means of doing it right in all regards.

I wonder how much other ecosystems can grow if they gave up the control and allowed their communities to drive progress. One of the greatest parts of python and rust is the ability to still innovate the needs of the community and not just corporate interests

thenerdhead | 9 months ago

I enjoyed reading this perspective, but I think it left out some crucial information. My thesis here is that the PSF - already woefully underfunded, with PyPI kept afloat by a massive in-kind donation that gets to stay off the books - grossly misallocates the funding it does receive. Further, there are obvious and serious gaps in the consideration they pay to the moral values they profess to hold most dear.

> The PSF currently employs 12 full-time staff members. Members of the board do not directly manage the activities of the staff—in fact board members telling staff what to do is highly inappropriate. Instead, the board is responsible for hiring an Executive Director—currently Deb Nicholson...

Ms. Nicholson is listed as both a board member and a staff member, and also an officer. Another staff member, Olivia Sauls, is also listed as an officer. Between officers, the board and staff, I count 25 unique individuals, who are presumably all paid.

> Many PSF activities are carried out by these volunteers, in particular via Work Groups.

The membership of these Work Groups also has heavy overlap. For example, four members of the Code of Conduct Work Group are on the Board as well - including Ms. Nicholson.

It's also worth noting that members of Work Groups are not necessarily elected. For example, the cited charter for the new "User Success" Work Group doesn't provide for the addition or removal of members at all, even though it's intended to operate perpetually. The Code of Conduct Work Group appoints its members, who may apply by private contact.

> You can see how that money has been spent in the 2023 Annual Impact Report.... The most significant categories of expenditure in 2023 were PyCon US ($1,800,000), our Grants program ($677,000), Infrastructure (including PyPI) ($286,000) and our Fiscal Sponsorees ($204,000)—I’ll describe these in more detail below.

I can't readily find these numbers in the report, but I can find that staffing for 2023 cost a total of about 1.3 million.

(The report for this year is, for some reason, only available in PDF format. Others pointed out that this was not great for accessibility - supposedly a core value of the PSF and several Work Groups - and this was agreed about, and then as far as I can tell nothing was done.)

> More recently, the PSF has started employing Developers in Residence to directly support the work of both the core Python team and initiatives such as the Python Package Index.

To my understanding, there are currently three of these: Łukasz Langa - the individual who implemented my ban from the Python Discourse forum - since 2021, and Petr Viktorin and Serhiy Storchaka since January.

That makes 28 likely people on the payroll by my count. Other core developers - dozens of them - operate purely on a volunteer basis. (In talks at PyCon, Raymond Hettinger used to joke about Guido van Rossum offering to double his $0 salary.)

> PyPI’s numbers are staggering. Today there are 570,000 projects consisting of 12,035,133 files, serving 1.9 billion downloads a day (that number from PyPI Stats). Bandwidth for these downloads is donated by Fastly, a PSF Visionary Sponsor who recently signed a five year agreement to continue this service.

Per the report, this amounts to over 600 petabytes of network traffic from PyPI per year. This would cost about 12 million dollars a year - nearly triple the operating budget - at market rates (https://aws.amazon.com/cloudfront/pricing/).

My experiments suggest that about a quarter of this could be eliminated simply by enabling and then mandating the use of LZMA (XZ) compression for sdists and wheels (which currently must both use ordinary Gzip compression). There is support for LZMA in the Python standard library. (The one-time cost of recompressing existing archives should be minuscule in comparison - granted that LZMA is computationally expensive, but currently the daily download volume is on the order of dozens of times the entire repository size.)

> The annual US Python Conference—PyCon US—is a big part of the PSF’s annual activities and operations. With over 3,000 attendees each year (and a $1.8m budget for 2023) running that conference represents a full-time job for several PSF staff members.... the importance of PyCon US to the Python community is such that the PSF is happy to lose money running the event if necessary.

For a sense of scale, this is 200 times the grant that made DjangoCon Africa possible - though it ran into serious issues due to underfunding (https://pythonafrica.blogspot.com/2023/12/an-open-letter-to-...).

Also for a sense of scale: the PyCon US Youtube channel (https://www.youtube.com/@PyConUS) has about 29k subscribers and a total of 1.35 million views across all videos. For comparison, that's not many more people than those taking the JetBrains Python developer survey each year (e.g. https://lp.jetbrains.com/python-developers-survey-2023/ claims 25k), and less than a sixth as many views as for the question "How do I list all files of a directory [in Python]?" on Stack Overflow (https://stackoverflow.com/questions/3207219/). But perhaps "the Python community" (with inclusivity as a core value) is meant to represent a much smaller group than Python developers in general.

zahlman | 9 months ago

So who manages Pypi? This document seemed vague on that. Maybe that's the problem with Pypi's progress in life.

Most packages on Pypi are complete crap. It's also heavily burdened with domain-specific applications and one-off student projects. They have no standards for what makes a useful package, and no ranking system aside from the number-of-downloads. I think package maintainers should be required to push an update every other year or have their package get dropped. I think frameworks should be separate from applications. I think packages without a lot of downloads should utilize endorsements and code-cleanliness metrics.

BrannonKing | 9 months ago

> This auditability is an interesting aspect of how 501(c)(3) organizations work, because it means you can donate funds to them and know that the IRS will ostensibly be ensuring that the money is spent in a way that supports their stated mission.

Come on. IRS is woefully understaffed. This might be a good talking point but short of some sort of egregious financial crime (like actual fraud) it seems unlikely the IRS gives a shit about how PSF spends its money.

pm90 | 9 months ago
[deleted]
| 9 months ago