GCP – Supporting the Python ecosystem

Python is critically important to both Google Cloud and our customers. It serves as a popular runtime for many of our hosted services, from the launch of App Engine more than a decade ago, to modern serverless products like Cloud Functions. We use the Python Package Index (PyPI) to distribute hundreds of client libraries and developer tools, including the popular open-source machine-learning library TensorFlow. And we use it internally as well, where it helps power many of our core products and services.

As part of our longstanding support for the Python ecosystem, we are happy to increase our support for the Python Software Foundation, the non-profit behind the Python programming language, ecosystem and community. 

First, we’re announcing a donation of more than $350,000 to support three specific PSF projects, with a focus on improving the supply-chain security of the Python ecosystem. These include:

• Productionized malware detection for PyPI

• Improvements for foundational Python tools and services

• A CPython Developer-in-Residence for 2021, who will work full-time to help the CPython project prioritize maintenance and address its backlog

Second, we have recommitted to our ongoing in-kind donation of Google Cloud infrastructure to the PSF. This helps support the critical infrastructure that the PSF operates, including the Python Package Index.

“Google Cloud has given us access to crucial peering agreements via Cloud Storage that allow us to cost effectively serve PyPI downloads while being good stewards of the limited resources we have from other infrastructure providers.” – Ee Durbin, Director of Infrastructure, Python Software Foundation

Third, the Google Cloud Public Datasets program is now the new home of a new public dataset of PyPI download statistics and PyPI project metadata, which we update in near-real-time. Anyone with a Google Cloud account can query these datasets with BigQuery, or with BigQuery sandbox, which offers up to 1TB/month of data queries for free. You can learn more about analyzing these datasets in this user guide.

“Publishing PyPI’s analytics as a public dataset on BigQuery has reduced the burden of supporting and managing access to information that has proven critical to maintainers of libraries as well as the team that keeps PyPI online. We are excited to realize even more cost savings and ease of deployment by moving the pipeline that feeds the dataset to Cloud Functions.” – Ee Durbin

And finally, as of last year, we have renewed our foundation sponsorship of the PSF for the eleventh consecutive year. Taking into account all these various forms of support, we are proud to become the organization’s first “Visionary” sponsor, the highest tier of their new sponsorship tier system. 

Like so many Google Cloud customers, we’re big believers in Python. Supporting the PSF in this way will help ensure that the Python ecosystem has a strong and viable future for many years to come. You can find more details about these projects and our sponsorship on the PSF’s blog.