Katie Dagon (Jun 06 2023 at 20:29): Katie Dagon (Jun 06 2023 at 20:29):ESIP Cloud Computing Cluster Session June 26: Dask and Coiled with Matt Rocklin of Coiled.io
Details here: https://discourse.pangeo.io/t/esip-cloud-computing-cluster-session-june-26-dask-and-coiled-with-matt-rocklin-of-coiled-io/3488
Deepak Cherian (Jun 26 2023 at 17:03):Happening now!
Join Zoom Meeting
https://esipfed-org.zoom.us/j/82186805451?pwd=SzNJbGloZjJHOTlFeGo2aWZjSEhGUT09
Meeting ID: 821 8680 5451
Passcode: ESIP23
Dask makes it easy to parallelize Python code, either your own custom code, or through integrations with common libraries (pandas, numpy, xgboost, xarray, etc.). While Dask is used in a broad set of applications, in this talk we’ll mostly focus on applications in the Pangeo ecosystem, a set of tools (Xarray, Dask, Zarr, and others) that make it easy to perform earth science applications at scale.
Coiled makes it easy to manage cloud infrastructure. Coiled came out of many experiments running Dask clusters on cloud infrastructure. We found that users had trouble managing software package versions, cloud credentials, managing idle clusters, managing users, and predicting and constraining costs. The Coiled platform was developed as a centralized service to deploy Dask clusters in a way that addressed these issues.
We hope to show that, with these two technologies, we’ve found a good way to enable the vast majority of Python users to run their analyses at scale.
Last updated: May 16 2025 at 17:14 UTC