AWS – PySpark available in AWS Clean Rooms

Today, AWS announces the general availability of PySpark in AWS Clean Rooms, enabling companies and their partners to run sophisticated analytics across large datasets using PySpark, the Python API for Apache Spark. With this launch, you and your partners can bring PySpark code and libraries to an AWS Clean Rooms collaboration and run advanced analyses without having to share underlying data or proprietary analysis methods. For example, an advertising measurement provider can use PySpark in AWS Clean Rooms to run their custom algorithms across multiple publisher datasets simultaneously to measure ad effectiveness. Similarly, a pharmaceutical company can run their proprietary algorithms and libraries across multiple healthcare provider datasets with appropriate patient consent to evaluate drug adherence across clinical trials, without sharing their proprietary data.

AWS Clean Rooms helps companies and their partners to easily analyze and collaborate on their collective datasets without revealing or copying one another’s underlying data. Companies can deploy their own clean rooms without having to build, manage, or maintain their own solutions or move data outside of their AWS environment. PySpark in AWS Clean Rooms is generally available in these AWS Regions. To learn more, visit AWS Clean Rooms.

Read More for the details.