Earth-Science | 2i2c

NASA Open Science ScienceCore tutorial available at github.com/sciencecore

Thu, 18 Dec 2025 00:00:00 +0000

As part of the NASA Open Science initiative, we co-developed a ScienceCore curriculum module with MetaDocencia that teaches researchers and educators how to use NASA Earthdata in the cloud to analyze climate risk.

You can find the material at github.com/sciencecore:

climaterisk: the Determining Climate Risks with NASA Earthdata Cloud module ( source repo).
scipy-2024-climaterisk and pydata-nyc-2024-climaterisk: conference tutorial versions of the climaterisk material.

We also run the shared sciencecore.opensci.2i2c.cloud JupyterHub and an opensci BinderHub that host the hands-on exercises. These deployments use JupyterHub and BinderHub for hosting live computational infrastructure, and Jupyter Book for hosting the reading materials.

Acknowledgements #

This work was supported by a NASA TOPS-T award that funded the ScienceCore climaterisk module and shared infrastructure for the broader ScienceCore community.

Thanks to MetaDocencia for leading translation and localization of the material into Spanish, and helping with content creation.

Thanks to Dhavide Arulia for leading much of the content creation.

Supporting NASA Openscapes Champions with Cloud Infrastructure

Sun, 30 Nov 2025 00:00:00 +0000

Openscapes ran a NASA Champions program in November, bringing 30 participants together to learn about NASA Earthdata and the earthaccess Python library. We provided JupyterHub infrastructure for hands-on breakout sessions - a good example of using shared infrastructure to facilitate learning and collaboration in remote events.

They used their JupyterHub for co-working, where participants practiced streaming techniques for accessing cloud data without downloading. Multiple NASA Data Centers (NSIDC, ORNL, ASDC, PO.DAAC) collaborated to co-teach using the shared environment, succeeding despite the event happening the day after a government shutdown.

They also used this to grow the OpenScapes community by getting attendees to join their slack and sign up for their December Earth Access hack day. It’s a great example of leveraging shared community infrastructure to help newcomers learn quickly and join a science community.

Read their full event summary to learn how they structured the program and engaged their community.

Communities learning from one another - Project Pythia and ICESat-2 Hackweeks

Tue, 21 Oct 2025 00:00:00 +0000

We wanted to share a short vignette about two of our communities learning from one another.

At the latest Project Pythia community meeting, Project Pythia met with representatives from ICESat-2 to share learning about notebooks and cookbooks in educational settings.

Anthony Arendt from UW’s eScience Institute shared how they’ve used educational notebooks in their hackweek programs. The discussion explored ways to improve cookbooks, especially for large collections that require different computational environments, sparking ideas about higher-level abstractions for organizing educational content. There is a lot of overlap in the needs and workflows of these communities, and we’re hopeful they can find ways to re-use one another’s ideas, content, and infrastructure.

One of our service goals is to make it easier for our member communities to learn from one another - using standardized tools and infrastructure means we can learn what works, what doesn’t, and collectively improve our workflows more quickly. We’re working on ways to encourage this kind of interaction in our member networks, so we wanted to celebrate this little win.

Learn more about these communities #

Project Pythia - An educational resource for geoscience computing with open-source Python
Project Pythia Cookbooks - Domain-specific example workflows for geoscience
ICESat-2 Hackweeks - Collaborative learning events combining tutorials, peer learning, and team projects

Harnessing Marine Open Data Science for Ocean Sustainability in Africa, South Asia and Latin America

Tue, 11 Mar 2025 00:00:00 +0000

Thank you to Emilio Mayorga for sharing this publication.

Several community members, including Paige Martin (Australian Climate Simulator), Eli Holmes (NOAA Fisheries), and Emilio Mayorga (University of Washington) published case studies in Oceanography magazine’s “Vision for Capacity Sharing” issue.

Their article Harnessing Marine Open Data Science for Ocean Sustainability in Africa, South Asia, and Latin America highlights the benefits of hackweek-style collaboration and learning events to build capacity in underrepresented communities, using 2i2c-supported JupyterHub for seamless set up and effective data sharing.

More on these three specific initiatives is available at their respective websites:

COESSING, Coastal Ocean Environment Summer School In Nigeria and Ghana.
OHWe - OceanHackWeek en Español (in Spanish).
ITCOocean Hack2Week (an Indian Ocean program). Training Course & HackWeek On Machine Learning Based Species Distribution Modeling.

We’re happy to see these communities extend their impact and make interactive computing more accessible to participants around the world.

Open infrastructure for collaborative geoscience with Project Pythia: Learning how to deploy a BinderHub on Jetstream2

Wed, 12 Feb 2025 00:00:00 +0000

Project Pythia and the “Jupyter notebook obsolescence” problem #

Project Pythia provides educational resources for essential software tools that enable open, reproducible and scalable geoscience, such as the Pangeo stack of packages (Xarray, Dask, Jupyter). Their Cookbooks are crowdsourced, community-curated, and open-source collections of Jupyter notebooks that demonstrate how to use these tools for cloud-native, geoscientific workflows (see our Project Pythia Cookoff blog post). However, “Jupyter notebook obsolescence” is a common problem: tutorials that were created a few years ago may no longer work due to changes in the software ecosystem and hampers the reproducibility of scientific results. A reproducible execution environment and the infrastructure to support it are essential for the long-term sustainability of these educational resources.

Leveraging NSF-funded cyberinfrastructure for BinderHub #

A BinderHub allows users to dynamically create custom computing environments from Binder-ready repositories containing computational notebooks and configuration files that describe the software environment required to run them. A public Binder service exists at mybinder.org (see our blog post about joining the mybinder federation 🎉) and is a successful example of how open cloud infrastructure can accommodate reproducible execution environments.

The resources available on such a public service are limited therefore 2i2c, together with Project Pythia, have been exploring how to deploy a BinderHub backed by larger resources from the NSF-funded cloud computing platform Jetstream2. This allows for larger simultaneous user loads, such as at workshops, as well as access to more powerful distributed and parallelized workflows required to process large geoscientific datasets, under a persistent resource allocation.

Learning how to deploy on OpenStack #

Jetstream2 uses OpenStack in order to manage pools of compute, storage and networking resources, and for our purposes we specifically make use of OpenStack Magnum Cluster API driver to manage Kubernetes for our deployment.

Cluster API needs a CAPI management cluster in order to manage other Kubernetes clusters, called workload clusters. On Jetstream2, this management cluster is gracefully created and operated by the Jetstream2 team, which means that the only task to worry about is creating and configuring the workload cluster.

For the workload cluster we used the Openstack Terraform provider to define the cluster template, the cluster itself and the node groups in a reproducible way.

After the cluster infrastructure was successfully created on Jetstream2, thanks to the 2i2c hub infrastructure being cloud agnostic as well, deploying BinderHub to Jetstream2, was a seamless experience and it was no different than on other cloud providers that we already supported.

We also learnt about some limitations of the Openstack Magnum driver project, which were expected given it being a relatively recent project, slowly being adopted, but they were all reported upstream and hopefully will soon be fixed.

Acknowledgements #

Jetstream2: Explore ACCESS allocation and Julian Pistorius for technical support
Thanks to Project Pythia for funding and collaborating with us on this work.
Andrea Zonca for preliminary work on Kubernetes deployments on Jetstream 2

NASA VEDA & 2i2c Update for Q4 2024 (Oct-Dec 2024)

Tue, 07 Jan 2025 15:18:37 -0800

A non-exhaustive list of things 2i2c and Development Seed did with the NASA VEDA project last quarter!

Automated backups and alerting with `jupyterhub-home-nfs` #

Tracking Issue

jupyterhub-home-nfs is a young project to provide flexible per-user home directory limits on JupyterHub - an important feature for controlling cloud costs. Tarashish Mishra and Sarah Gibson have been leading this project for the last few months. Since we are moving away from AWS Managed EFS here, we had to do some work to recreate some of the benefits EFS gives us out of the box. During this quarter, we:

Set up automated backups so we can recover files in cases of disaster
Set up automated alerting (via prometheus and pagerduty) to know if our backing EBS device is getting full and we need to perform a manual intervention
Deployed this to a few other communities ( CryoCloud and NMFS Openscapes) to broaden adoption.

We will continue doing work on jupyterhub-home-nfs in the upcoming quarter! If this is functionality you are interested in deploying, please reach out to us to collaborate!

Enable users to dynamically build environments with `jupyterhub-fancy-profiles` #

Tracking Issue

We covered this more extensively in another blog post, so go read that!

This work in particular is a good demonstrator of 2i2c’s value - it started off with a grant from GESIS, and now with support from NASA IMPACT we are able to bring it to a lot of communities, not just the ones that funded it.

Ongoing work here will focus on improving the UX as well as better documentation so users can actually use it!

“Open in QGIS” from VEDA UI #

Tracking Issue

We had worked in the past with many communities in enabling QGIS on the Cloud, and this quarter we got closer to enabling a contextual ‘Open in QGIS’ button in the VEDA Dashboard! Here is a quick demo:

(This shows the workflow when user is already logged into the JupyterHub and had started the server)

You can play with this in this preview, although you need to have access to the NASA VEDA hub to fully try it out at this point.

Tarashish from Development Seed is again responsible for most of the work here, available in jupyter-remote-qgis-proxy. You can use it to create ‘magic links’ that will open QGIS in a desktop environment in your browser, and add a specific layer to it! Our hope is that this allows primarily GIS folks to better use tools they already are familiar with in cloud based contexts.

Other updates #

We participated heavily in an evaluation process for the authentication and authorization solution to be used across NASA VEDA! Tracking Issue
We are very close to rolling out JupyterHub 5.0 and associated changes across all our hubs, which will enable us to eventually offer per-group shared directories! Tracking Issue

Acknowledgements #

Thanks to the NASA VEDA project for thir ongoing support for this work.
Thanks to DevSeet for their collaboration and leadership on this project.

2i2c communities at AGU 2024

Mon, 09 Dec 2024 00:00:00 +0000

We are proud to share that several of 2i2c’s community partners are presenting their work at AGU 2024! In each case, 2i2c’s infrastructure plays a part in helping communities create and share knowledge, and grow their community. As an organization rooted in community-centric practices, we are particularly excited to see 2i2c represented “indirectly” at this conference, and to see ourselves as a supporting role enabling the impact of others.

Here’s a summary and links to all of the sessions. See below for a brief overview of seach one.

ED31G-2272 Breaking down the barriers to Open Science with Project Pythia #

Link to session

Hall B-C (Poster Hall) (Convention Center)

Abstract #

Project Pythia is an open access educational initiative established with funding from the U.S. National Science Foundation. Its mission is to help students and scientists enhance their skills and adopt best practices using the tools and technologies of open science. As part of the Pangeo community, Project Pythia primarily focuses on the Pangeo stack, which includes cloud computing, Jupyter technologies, GitHub, and various software packages in the Scientific Python ecosystem, centered around Xarray. Project Pythia offers a wide range of open access content, such as datasets, software, tutorials, and annotated real-world workflows presented in the form of Jupyter Books.

Project Pythia serves as a resource for scientists, promoting and fostering open science. Although it is not a scientific research artifact itself, the development of Project Pythia adheres to many best practices advocated by open science proponents. The Pythia team actively encourages community engagement and collaborates openly with scientists and technologists to create new content. All Pythia resources are freely accessible, and the project follows the FAIR principles (Findable, Accessible, Interoperable, and Reusable) for managing research outputs, including publications, data, and other materials. We support and facilitate open evaluation and peer reviews of content to ensure verifiability and trust. Lastly, we endeavor to openly discuss ideas, designs, and methods before implementation.

This presentation will provide an overview of Project Pythia’s extensive educational resources and share our experiences in applying many open science principles to develop this flagship training resource for the geoscience community.

Authors #

John Clyne - NSF National Center for Atmospheric Research (first author)
Drew Camron - University Corporation for Atmospheric Research
Orhan Eroglu - NSF National Center for Atmospheric Research
Robert Ford - University at Albany State University of New York
Julia Kent - NSF National Center for Atmospheric Research
Ryan May - University Corporation for Atmospheric Research
James Munroe - 2i2c / Code for Science and Society
Brian E J Rose - SUNY at Albany

ED31G-2277 PACE Hackweek: An open community keeping up with PACE #

Link to session

Abstract #

The NASA Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission, while bringing NASA’s Earth System Observatory up to speed with aquatic, atmospheric, and terrestrial science capabilities, is also providing data records of the Earth System for the next generation of scientists to grow into. The goal of the PACE Hackweek, supported by the Ocean Carbon & Biogeochemistry program and hosted at the University of Maryland Baltimore County in August 2024, was to enrich and support the practice of open science by both emerging and established researchers. Cloud-compute resources for the event were provided by CryoCloud, a NASA funded collaboration between ICESat-2 and the International Interactive Computing Collaboration (2i2c) to provide cryosphere researchers with a shared JupyterHub. We, the hackweek mentors, were buoyed by the NASA Openscapes program and adopted its mantra of striving toward “a kinder science for future us.” Participants faced two novelties: the “firehose” of data from the PACE instrument array (a hyper-spectral imaging spectrometer, a wide-swath hyper-angular polarimeter, and a narrow-swath spectro-polarimeter), and the distribution of PACE collections through the NASA Earthdata Cloud (a first for the Ocean Biology Distributed Active Archive Center). We present our approach and the challenges undertaken to hold an in-person, social coding event with 45 participants that provided a collaborative, supportive launchpad for doing open science with PACE. All lectures and tutorials produced for the event are freely available for examination and reuse. Our results additionally include highlights from the demonstration projects pursued by event participants and results from two post-event, qualitative surveys. One anonymous survey gathered participant feedback that will inform plans for growing these 45 participants into a lasting, open community. A separate, anonymous survey recorded participant demographics in order to evaluate our efforts at increasing diversity within the community of PACE data users. Key points of discussion include participant views, informed by our event, on whether and how the NASA Earthdata Cloud is a significant resource for the practice of open science with PACE, and how a shared JupyterHub can further the practice of open science by the community it serves.

Authors #

Ian Carroll - NASA Goddard Space Flight Center; University of Maryland Baltimore County (first author)
Kelsey Bisson - NASA Headquarters
Sean Foley - NASA Goddard Space Flight Center; Morgan State University
Patrick Clifton Gray - University of Maine
Elizabeth E Holmes - NOAA Fisheries
Carina Poulin - Science Systems and Applications, Inc.; NASA Goddard Space Flight Center
Tasha Snow - NASA Goddard Space Flight Center; University of Maryland College Park
Guoqing Wang - NASA Goddard Space Flight Center; Science Systems and Applications, Inc.
Jeremy Werdell - NASA Goddard Space Flight Center
Anna Windle - NASA Goddard Space Flight Center; Science Systems and Applications, Inc.
Pengwang Zhai - Department of Physics, University of Maryland Baltimore County

IN13A-2147 Including more solutions and more solvers via actionable open science #

Link to session

Monday, 9 December 2024 13:40 - 17:30
Hall B-C (Poster Hall) (Convention Center)

Abstract #

If we’re asking people to change for open science, we must be willing to change ourselves. Internalizing this as individuals and institutions is critical - “to address our climate emergency, we must rapidly, radically reshape society. We need every solution and every solver” (Johnson & Wilkinson, All We Can Save).

Radically reshaping our society and including more solvers requires Earth scientists of all disciplines, across AGU, to work together in new ways. Many of these shifts can be considered Open science. They change how we work daily, not just the open products we produce. And for that, people need to consider themselves part of a team, let go of perfection and embrace a growth mindset to continually reflect and improve skills – no matter their job title. Further, open science requires all of us to see ourselves as leaders making small changes that collectively add up to a movement.

Openscapes is an open source approach to cultivating leaders and change makers. Openscapes’ flywheel approach intervenes and builds momentum through identifying mentors within organizations and mentoring teams curious about shifting to open science (Robinson & Lowndes 2022). Collectively, the Openscapes flywheel iterations have had a significant impact over the past five years across institutions like the federal government and academia that seem impossible to change. Through stories working with professional scientists over the past 5 years including at NASA, NOAA, Black Women in Ecology, Evolution, and Marine Science, and many universities, and open source software communities like 2i2c, Posit, Pangeo, and RLadies, we will share actionable insights for flourishing in the open science commons, and are interested in learning with and growing flywheel momentum further at AGU.

Authors #

Julia S. Stewart Lowndes - Openscapes, LLC (first author)
Ileana Faye Fenwick - Hampton University
Elizabeth E Holmes - NOAA Fisheries
Luis Alberto Lopez - National Snow and Ice Data Center (NSIDC), CIRES, University of Colorado Boulder
Erin Robinson - Self Employed

IN34A-01 Beyond Open Data: Ensuring True Accessibility for All (Invited) #

Link to session

Abstract #

The Earth Observation (EO) industry has seen rapid technological advancements alongside a massive increase in the number of private and public missions, leading to exponentially growing data archives. For publicly funded entities, this data is typically required to be freely available. However, open data does not always guarantee accessibility, and significant barriers remain for even the most advanced users. The stagnation in the use and adoption of open data can be attributed to several factors, including 1) challenges in unifying and maintaining metadata standards, 2) inefficiencies associated with legacy data formats, 3) a lack of training and resources for transitioning to cloud-based infrastructure, and 4) systemic social inequalities.

This talk will explore real-life examples of these barriers and highlight success stories that have emerged from partnerships largely originating within open-source communities which foster diverse connections between private and public entities including efforts like GeoZarr, pangeo-forge, Openscapes and 2i2c. While many advancements in improving the usability and accessibility of EO data have come from private efforts (i.e. Google Earth Engine), the shutdown of the Planetary Computer is a reminder of the need for publicly funded alternatives. The sustainability of open source projects will be addressed, with questions posed around reliable funding mechanisms as a means to ensure equitable development to address barriers and ensure accessibility for all. While this talk will be presented by one individual, it is the review and reflection of the work done by dozens of people across various organizations.

Authors #

Brianna Rita Pagán - NASA Goddard Space Flight Center; ADNET Systems Inc. Greenbelt (first author)

Introducing GeoLab - An EarthScope JupyterHub for Enabling Collaborative Cloud-Native Geophysical Data Analysis and Skill Development Workshops #

Link to session

Abstract #

The EarthScope Consortium manages NSF’s GAGE and SAGE facilities and makes all of its geophysical data available in a commercial cloud system. This enables EarthScope and the communities it supports to leverage the abundant computational resources and cost-effective benefits of adopting data-proximate workflows with direct access to large, analysis-ready geophysical data sets.

In recent years, JupyterHub environments have gained popularity with data enthusiasts for their ability to provide open access to powerful compute resources. As part of a broad effort to support communities with intuitive resources to quickly adapt their workflows to the cloud, EarthScope has partnered with 2i2c to operate a scalable JupyterHub environment in AWS that will provide equitable access to cloud compute resources for researchers, educators, and the general public. GeoLab, the EarthScope hub, is aligned with related open science initiatives to establish rigorous and transparent standards for reproducible, data-intensive workflows. In addition to promoting interdisciplinary and inter-institutional collaborative work between researchers in GeoLab, EarthScope is developing and hosting workshops that can support both in-person and asynchronous learning modules that will train users how to utilize these new resources and transition their work to the cloud. We are excited to invite all geophysical data users to participate in the vigorous growth of this new platform and collaborate with adjacent open-science compute hub initiatives.

Authors #

Robert T Weekly - EarthScope Consortium (first author)
Tammy K Bravo - EarthScope Consortium
Jerry A Carter - EarthScope Consortium
Tim Dittmann - EarthScope Consortium
Alex Hamilton - EarthScope Consortium
David Mencin - EarthScope Consortium
Chad Trabant - EarthScope Consortium
Sarah Wilson - EarthScope Consortium

Link to session

Abstract #

Project Pythia is the flagship education and training initiative of the Pangeo community. Pangeo has advanced transformative platforms and paradigms for “Big Data” geoscience in the cloud; Pythia is creating on-ramps for new users with open, interactive learning resources centered on Python in the geosciences. Pythia is now building a vibrant community-owned clearinghouse of accessible, reusable, and reproducible tutorials and exemplar workflows in the cloud known as Pythia Cookbooks.

“Cookbooks” imply collections of recipes for transforming raw ingredients (publicly available data) into scientifically useful results. Based on Jupyter notebooks, Cookbooks are explicitly tied to reproducible computational environments and supported by a rich cloud-based infrastructure enabling collaborative authoring and automated health-checking – essential tools in the struggle against the widespread notebook obsolescence problem. Cookbooks are hosted on Pythia’s searchable gallery and nurtured by a growing community of open science enthusiasts from across the geosciences. The Pythia Cookbook gallery is essentially a crowd-sourced, community-curated collection of best practices for data analysis and visualization.

Here we will outline the stack of technologies and infrastructure enabling cookbook creation, collaboration, testing, publication, and interactive deployment, and how these are used in service of building an inclusive participatory community. We will discuss existing technical and social hurdles for contributors, as well as new infrastructure developments in collaboration with the Executable Books Project that are reducing these hurdles.

Authors #

Brian E J Rose - University at Albany State University of New York (first author)
Drew Camron - University Corporation for Atmospheric Research
John Clyne - NCAR
Orhan Eroglu - NSF National Center for Atmospheric Research
Robert Ford - University at Albany State University of New York
Julia Kent - NSF National Center for Atmospheric Research
Ryan May - University Corporation for Atmospheric Research
James Munroe - 2i2c / Code for Science and Society
Kevin Tyle - SUNY at Albany

U13A-2350 Supporting NASA Earthdata users in the Cloud: NASA Openscapes JupyterHub and User Onboarding & Fledging #

Link to session

Abstract #

In this talk we will highlight our NASA Openscapes community teaching approach to using the 2i2c-managed JupyterHub – how we’ve collaboratively developed it to meet user needs, and how it continues to enable researchers and users of NASA Earthdata in the new Cloud paradigm.

NASA Openscapes is an open source mentor community across NASA Earth science data centers ( DAACs) that helps users explore and use the Cloud for their science and applications. Earthdata Cloud Cookbook is a learner-focused open source tutorial collection that we update openly as we learn together.

A critical piece of the NASA Openscapes effort is our NASA Openscapes 2i2c JupyterHub, a managed cloud computing space. By working with cloud early adopters and science Champions, responding and co-developing solutions, the JupyterHub has evolved since its early days in 2021. We support cloud computing for several languages (python, R, Matlab, QGIS) and common science libraries with corn; we streamlined how to bulk-add workshop participants via GitHub Teams; we established policy and technology for a special authentication mechanism for large scale workshops; we are developing earthaccess as an community-developed python library for NASA Earthdata search and access, whether locally or in the cloud.

Based on the last 3 years of engaging with the user community and the Hub, we have evolved how we onboard (first experience in the cloud) and fledge (set up for Cloud that includes a plan, how to do it, how to pay for it; leaving the nest and perhaps building your own). Fledging is an important part of adoption and initiating users to the Cloud - where do researchers go when they decide to do their science in the Cloud? We’ve been developing practices that aim to be equitable and consider policy (cost), technical (where do people go, what admin setup is needed, what tech like base images etc are needed), and social (how do I learn, get support) aspects, and look forward to discussing further at AGU.

Authors #

Julia S. Stewart Lowndes - Openscapes, LLC (first author)
Andrew Barrett - National Snow and Ice Data Center (NSIDC)
Carl Boettiger - University of California Santa Cruz
Aaron M Friesz - Organization Not Listed
Elizabeth E Holmes - NOAA Fisheries
Alexis Hunzinger - NASA Goddard Space Flight Center
Daniel Kaufman - NASA Langley Research Center
Luis Alberto Lopez - National Snow and Ice Data Center (NSIDC), CIRES, University of Colorado Boulder
Catalina M Oaida Taglialatela - NASA Jet Propulsion Laboratory, California Institute of Technology
Yuvi Panda - University of California, Berkeley
Michele Thornton - Oak Ridge National Laboratory

V31A-08 VICTOR – A new Cyber-infrastructure for Volcanology #

Link to session

Abstract #

Numerical models are essential for forecasting volcanic hazards for both short-term responses and long-term hazard assessment. While many models of volcanic processes already exist, challenges in finding, installing, and evaluating these models, coupled with limited computational resources, hinder their widespread use. To address this, we introduce VICTOR, the Volcanology Infrastructure for Computational Tools and Resources.

VICTOR is a cutting-edge cyber-infrastructure platform offering an open-source, cloud-based environment tailored for the volcanology community. It features Jupyter notebooks that integrate existing volcano models, such as the lava flow codes MOLASSES and IMEX_lava, the tephra and ash dispersal codes Tephra2 and HYSPLIT, and the mass flow code TITAN2D. The backend of VICTOR is managed as a JupyterHub, operated by the non-profit 2i2c under the Code for Science and Society.

VICTOR not only provides access to individual modeling tools, but also hosts workflows that use them in data inversion, model benchmarking, and uncertainty quantification. For example, we developed a workflow to validate mass flow models using multiple metrics and Bayesian statistics. VICTOR provides built-in access to external databases such as OpenTopography, Copernicus, and NASA’s remote sensing products to streamline obtaining and using data in workflows.

VICTOR also serves as an educational resource. In Spring 2023 and 2024 we taught graduate level, multi-institutional courses in Computational Volcanology using VICTOR, and we are creating multilingual tutorials for the workflows. We are developing teaching modules on topics such as lava flows and remote sensing to be shared with instructors. Lastly, VICTOR collaborates with national efforts including CONVERSE and SZ4D.

In summary, VICTOR addresses the critical need for accessible, effective volcanic hazard modeling tools and resources, fostering advancements in both research and education within the volcanology community.

Plain-language Summary #

VICTOR is a new online platform designed to help scientists predict volcanic hazards more easily. Traditional models can be difficult to find, use, and combine with other tools. VICTOR solves these problems by offering a cloud-based, open-source environment with user-friendly tools.

VICTOR includes tools like Jupyter notebooks that combine various volcano models for lava flows, ash dispersal, and mass movements. It operates through JupyterHub, managed by the non-profit 2i2c. The platform not only provides access to these models but also offers workflows for tasks like data analysis and model validation. For example, it has a workflow for testing mass flow models using multiple evaluation methods.

VICTOR simplifies data access by connecting directly to databases like OpenTopography and NASA’s remote sensing products. It’s also an educational tool, used in graduate courses and offering multilingual tutorials. Additionally, VICTOR is developing teaching materials on topics like lava flows and remote sensing and collaborates with national projects like CONVERSE and SZ4D.

In essence, VICTOR makes volcanic hazard modeling more accessible and effective, benefiting both research and education in volcanology.

Authors #

Einat Lev - Columbia University of New York (first author)
Sylvain J Charbonnier - University of South Florida
Charles Connor - University of South Florida
Samuel Krasnoff - Lamont -Doherty Earth Observatory
Abani K Patra - University at Buffalo

From the GeoLab team - Pancakes are the future of geophysical data processing

Tue, 01 Oct 2024 00:00:00 +0000

Our partner EarthScope published a blog post about their cloud stack which includes heavy mention of 2i2c’s infrastructure model. Check it out!

Keeping PACE with GPU enabled compute to detect global cloud cover using satellite data

Mon, 12 Aug 2024 00:00:00 +0000

(left, b&w) Model inputs and (right, color) model outputs of a simple multi-layer perceptron for detecting cloud cover.

PACE is the NASA Plankton, Aerosol, Cloud, ocean Ecosystem mission that focuses on understanding ocean health and its impact on the atmosphere. Together with the Ocean Carbon and Biochemistry (OCB) program, a one-week hackathon ran from Aug 4 to Aug 8 on the 2i2c-hosted CryoCloud hub. The goal of the hackathon was to explore new Earth science data streams provided by the OCI, SPEXone and HARP2 instruments using Python.

Machine Learning with GPUs #

One of the most advanced tutorials delivered during the hackathon was the Machine Learning Tutorial. The tutorial focused on creating a machine learning pipeline to detect cloud cover from satellite imagery. This was done by training a convolutional neural network (CNN) to assign each pixel a binary value to indicate whether the location was covered by cloud or not. To improve the spatial context beyond a single pixel value, as the likelihood of a pixel containing cloud cover increases if its neighbours also contain cloud cover, the CNN needs to be trained on the entire image at once rather than at a single pixel level. This massively increases the training time, but also allows the CNN to learn more complex relationships between pixels.

GPUs have a far greater number of cores than CPUs that are well-suited for accelerating the massive parallel processing needed to train a neural network on the large amounts of image data in the above scenario. PyTorch is a popular Python library for training CNNs, available for both CPUs and GPUs, and is an ideal tool for performing this kind of work. In terms of the accelerator hardware available on the CryoCloud hub, 2i2c provisions an instance with an NVIDIA Tesla T4 GPU with 4 CPUS, 16GB of RAM and 2,560 CUDA cores.

Managing shared memory on 2i2c hubs #

While developing the above tutorial, tutorial lead Sean Foley (NASA/GSFC/SED & Morgan State University & GESTAR II) noticed that training neural networks was way slower than it should be given the GPUs available to them. They investigated the issue, and with help from the 2i2c engineering team, it was determined that shared memory was the issue. PyTorch uses shared memory via /dev/shm for faster parallel processing, and maximizing use of GPU. However in containerized environments, this is limited to a maximum of 64MB by default.

You can check the amount of shared memory available on your hub in a terminal with the command

df -h | grep /dev/shm

As you might expect, 64 MB of shared memory is not enough for training over 160,000 images in the tutorial. 2i2c was able to remove the limit, making /dev/shm share the memory the user has selected via their profile list, rather than be artificially limited to any particular size. This was done for all users on the CryoCloud hub within an hour of the issue being reported and we upstreamed the change for all 2i2c hubs (see GitHub pull requests for CryoCloud and all 2i2c hubs).

Conclusion #

This event demonstrates the economy of how running shared and open infrastructure dynamically solves problems for the benefit of many users, not just for one occasion. Learning experiences such as the above are transferred and embedded upstream into transparent and flexible open source software that impacts not only all users of 2i2c operated hubs, but also generalized for the wider research community at large (case in point, see the Slack thread below from Eli Holmes, operator of the NOAA Fisheries hubs)! We are grateful for the strong partnerships with our communities who help us to co-design impactful solutions that are specific for their needs and accessible to all.

The power of open infrastructure beyond 2i2c-operated hubs

References and Acknowledgments #

Sean Foley (NASA/GSFC/SED & Morgan State University & GESTAR II)
Tasha Snow (ESSIC UMD & NASA GSFC & CryoCloud)
PACE Hackweek Jupyter Book

Ephemeral Interactive Computing for the AmeriGEO Workshop

Mon, 05 Aug 2024 00:00:00 +0000

AmeriGEO provides a framework for cooperation in the Americas for the use of Earth data to benefit science and society with data driven decision-making. As part of a virtual workshop held on 1st August 2024, 2i2c provided an interactive computing environment to support the delivery of a NASA Open Science / ScienceCore water module.

The workshop was facilitated by Kytt MacManus and Juan F. Martinez (both of CIESIN, Columbia University, New York). Juan presented interactive R code, with explanatory content written in Quarto, for hotspot vulnerability analysis for floods and landslides focused on Ecuador using earth observation data and socioeconomic data to develop an index of vulnerability. Check out their GitHub repo and explore interactively yourself on our BinderHub (see our blog post for more details of how to access the BinderHub deployments to provide Ephemeral Interactive Computing for NASA Communities).

Over 100 participants were able to access the interactive workshop on our infrastructure, with 8 GB of RAM per user to facilitate the processing of large amounts of earth data. We are pleased that the workshop was successful and the platform was able to provide a great experience for participants. After the workshop, Kytt reported that

The technology worked as expected and we didn’t run into 1 major technical problem. Thank you so much for that!".

Acknowledgements #

Funding from Science Mission Directorate’s Open Source Science Initiative, Research Opportunities in Space and Earth Science (ROSES-2022), F.14 Transform to Open Science
NASA NSPIRES F.15 High Priority Open-Source Science Award NNH22ZDA001N-HPOSS
NASA Socioeconomic Data and Applications Center (SEDAC)
Kytt MacManus
Juan F. Martinez
James Munroe for providing support and assistance for setting up the cyberinfrastructure for this workshop.

Collaborating with Development Seed to deliver cyberinfrastructure for NASA VEDA

Fri, 12 Jul 2024 00:00:00 +0000

Thank you to Sajjad Anwar and Sanjay Bhangar for contributing to this post.

The VEDA dashboard

The 2i2c team are proud to continue our strong working collaboration with Development Seed, following our previous work on launching the US GHG center (also see the Development Seed blog post). Together with scientists at NASA in our regular sync touchpoints, we have recently delivered a tranche of improvements to the Visualization, Exploration and Data Analysis (VEDA) project.

This platform is designed to thread open-source components together to consolidate GIS delivery mechanisms, processing, analysis and visualization tools, and presented in a collaborative interactive computing environment. All code repositories and associated resources stemming from this work are available on the VEDA GitHub page.

In the spirit of fully open development, you can see the objectives the combined 2i2c and Development Seed team had for the last quarter. In this blog post, we will describe some of the significant ones!

Better image management and testing #

The repo2docker-action is a GitHub action simplifying image building and testing for use with JupyterHub, using either a Dockerfile or various configuration files (like requirements.txt, environment.yml, etc) supported by repo2docker. We migrated our image building pipeline from a somewhat homegrown solution to this upstream action, making image updates and testing much easier. In particular, we can automatically run test notebooks on every change we make to the image! This way, we can easily catch any breaking changes in library versions or other package installs without disrupting users. We also debugged and contributed upstream fixes to the testing infrastructure so everyone could benefit from this, rather than just us.

Automatically pulling example notebooks on startup #

When a user logs into a JupyterHub, it is very helpful if we could have a bunch of example notebooks and other content pre-populated for them so they can get started right away. nbgitpuller is heavily used for this particular use case. However, it requires that nbgitpuller is installed inside the image the user is using - and not all images have it installed. In particular, we wanted to continue using the (wonderful) Rocker images maintained upstream for R users, however they do not have nbgitpuller installed. To solve this problem we built jupyterhub-gitpuller-init, which can be used as an init container to pre-populate user content on persistent home directories regardless of the image used. We also made sure to build this in a way that anyone can use it, and it is not tied into either 2i2c or VEDA infrastructure!

Opening specific visualizations in QGIS via URL #

QGIS is the world’s most used open source GIS software, and previously 2i2c had worked with Openscapes and QGreenland to bring this desktop software to JupyterHub. We had previously worked on a container image that allows users to access large datasets stored in the cloud directly through QGIS on the JupyterHub, allowing users to work with much larger datasets than they could on their desktops by bringing cloud compute adjacent to the data. As a continuation of this work, we developed jupyter-remote-qgis-proxy, which builds QGIS specific features on top of jupyter-remote-desktop-proxy. In particular, it allows creation of shareable links that when clicked, opens specific datasets and layers in QGIS in a JupyterHub! You can see this in action:

Launching QGIS on a Linux desktop served by the VEDA JupyterHub

This opens up exciting future possibilities. Imagine this exploration of the Camp Fire having an ‘Open in QGIS’ button that enables further exploration of the data without the user needing to download or install anything! Work will continue in the coming quarter towards achieving this vision.

We are also excited to see recent work in this space from QuantStack and Simula Labs, and will follow up to ensure an orderly transition to more web native workflows for existing users of QGIS in due time.

Better Profile Selection #

This is a continuation of our GESIS collaboration. In the path to deploying dynamic image building to end users, we wanted to stabilize jupyterhub-fancy-profiles enough to deploy to users of VEDA (and eventually everyone else). This is the primary interface users see after they log in to JupyterHub, and was ripe for UX improvements. The default interface looks like this:

The revamped one is much more streamlined and looks like this:

Revamped Profile Screen

This is currently deployed to a staging hub and has helped us shake out a lot of bugs! We expect the improved interface will be rolled out to all users in the near future. We are also planning further development to make the user experience even better and smoother for everyone.

Supporting workshops #

End users benefiting from our work is what ultimately gives meaning to our work. To that end, we were very happy to support running workshops during this collaboration – see our related blog post US Greenhouse Gas Center supports summer school at CIRA for more information.

Ongoing Collaboration #

Delivering on these objectives in a timely way heavily depended on the success of the team collaboration. Sanjay Bhangar of Development Seed commented

Working closely with the 2i2c team on growing features to support users on the VEDA and GHG Center hubs has been absolutely amazing. With 2i2c’s deep experience in the Jupyter ecosystem, we have been able to implement some fairly complex features quite easily, and their strong open-source roots have ensured that whatever we work on is broadly useful to the wider Jupyter and scientific computing communities.

Take a look at the companion Development Seed blog post of this work.

This collaboration continues, and we have now published our objectives for the coming quarter. Watch this space!

Acknowledgements #

Development Seed
NASA IMPACT
Tarashish Mishra, Julia Signell, Oliver Roick, Slesa Adhikari and Sanjay Bhangar for various code contributions towards these objectives

Openscapes Host a Surface Biology and Geology Workshop with Shared Password Feature

Tue, 09 Jul 2024 00:00:00 +0000

Thanks to Brianna Lind, Julia Lowndes and Andy Teucher for contributing to this blog post!

Surface Biology and Geology: VITALS Workshop

Openscapes is a value-based initiative that supports kinder, better science based on open source community. NASA Openscapes is in its fourth year as a project supporting NASA Earth science in the Cloud, co-developed by Julia Lowndes (Openscapes) and Erin Robinson (Metadata Game Changers).

The initiative recently supported the Surface Biology and Geology: VITALS Workshop hosted by NASA Land Processes Distributed Activate Archive Center (LP DAAC) and NASA Jet Propulsion Laboratory (JPL).

Instructors used the 2i2c Openscapes Hub to lead hands-on exercises teaching learners how to manipulate data collected from the ECOSTRESS and EMIT instruments onboard the International Space Station. They used Jupyter notebooks in the Hub to demonstrate how open source tools together with cloud data and compute resources could effectively analyse the the Canopy Water Content and the Land Surface Temperature over the Jack and Laura Dangermond Preserve, Santa Barbara, CA.

Plot of the Canopy Water Content over the Jack and Laura Dangermond Preserve, Santa Barbara, CA from a VITALS Workshop Jupyter notebook.

This event was attended by around 250 participants. An event of this size therefore requires a frictionless login flow so that organizers could focus on the essential complexity of teaching data analysis rather than the accidental complexity of managing Hub authorization. GitHub authentication is the default option for most 2i2c Hubs for research use cases, but for an educational event of this size this option was not fit for purpose since organizers had to

Retrieve the GitHub usernames of each participant (assuming everyone was familiar with GitHub!)
Manually invite GitHub users to a GitHub organization to authorize access to the Hub (invitations would expire within seven days)
Repeat the above two steps last-minute for participants who showed up on the day without preparing
Manually remove GitHub users from the GitHub organization if they wanted to revoke access to the Hub after the event.

In response to this need, we developed a shared password feature so that workshop organizers can simply hand the share password out to learners for access to the Hub. This bypassed the manual labour of managing GitHub accounts while not adding to the learner’s high cognitive load and improving the participant’s learning experience overall.

One of the elements that enabled us to recognize and solve this issue effectively is our close partnership with the Openscapes team. We engage in regular 6-weekly catch-ups where we can learn about user requirements and how we can develop our infrastructure to co-create optimal solutions. Together with our Product Delivery Flow, we were quickly able to architect the shared password solution in time for the workshop.

Feedback from Brianna Lind (LP DAAC)

We have documented the technical infrastructure changes required to enable a shared password for the Hub in our Infrastructure Guide and hope to support many future events with this mechanism!

Acknowledgements #

NASA Openscapes
NASA LP DAAC
NASA JPL
NASA ROSES funding
NASA Open Science / ScienceCore for supporting some of our work on JupyterHub.

Determining Climate Risks with NASA Earthdata Cloud at Scipy 2024

Mon, 08 Jul 2024 00:00:00 +0000

Determining Climate Risks with NASA Earthdata Cloud is a ScienceCore curriculum module that comprises part of NASA’s Open Science and Transform to Open Science (TOPS) initiatives. The aim of this module is to deliver a hands-on experience with “data-proximate computing” in the cloud with NASA Earthdata products with content co-developed with MetaDocencia.

This module was delivered as a SciPy tutorial at this year’s conference. 2i2c have been working closely with the organizers to provide the hub infrastructure for the tutorial, including enabling a shared password for easy authentication (see our Openscapes post for more about this feature) and operating a small binder service for participants to view content after the event.

You can take a look at the tutorial on the NASA ephemeral hub!

The event was well-attended, with 40 learners taking part. Special thanks go to the organizers Dhavide Aruliah, Karthik Venkataramani and Patricia A. Loto for leading the tutorial.

Acknowledgements #

NASA F.14 Transform to Open Science Training award NNH22ZDA001N-TOPST (80NSSC23K0861), which seeded this contribution to NASA ScienceCore
MetaDocencia
Dhavide Aruliah
Karthik Venkataramani
Patricia A. Loto

Ephemeral Interactive Computing for NASA Communities

Thu, 27 Jun 2024 00:00:00 +0000

We are pleased to announce that we have deployed two ephemeral hubs for NASA communities!

What did we do? #

As part of the deliverables for our NASA High Priority Open-Source Science (HPOSS) award, we deployed two new ephemeral hubs:

a public small BinderHub that offers a “reader” experience where learners can interactively view GitHub repositories that deliver light scientific content with small compute and no barriers to authentication
a big BinderHub that offers an “explorer” experience where learners can log in to access more substantial compute resources to interactively investigate large datasets and run large workflows from any GitHub repository.

These services enrich the interactive computing ecosystem for NASA communities by

improving the shareability and reproducibility of scientific information
broadening participation for historically excluded and under-resourced science communities
enabling activities such as hackathons, demonstrations or training, during workshops and conferences.

How did we do it? #

Ephemeral interactive computing services benefited from some of our previous development work in collaboration with GESIS (see our detailed blog post for more information). The research and development of this project achieved wide-reaching impact across many NASA communities we currently serve, including ScienceCore, Openscapes, US Greenhouse Gas Center, VEDA and CryoCloud; as well as networks beyond the NASA scope, such as the NSF-funded Project Pythia and HHMI-funded Spyglass projects.

What next? #

We will focus on bolstering the community- and knowledge-building needed for making the best use of these binder services in the next phase of our HPOSS work to accelerate broader participation in science. This includes opportunities such as running workshops and tutorials, as well as disseminating best practices for collaborative research. Further engineering developments will proceed in collaboration with the NASA VEDA project to set up a binder service, improve the sharing of custom environments, and refine QGIS integrations for analysing geospatial data.

Can I use this ephemeral hub service? #

The answer is yes!

For the public small BinderHub anyone can view GitHub repositories that deliver light scientific content with small compute and no barriers to authentication
For the big BinderHub you will need to be member of a NASA community. This requires a GitHub account for membership of the GitHub Team 2i2c-nasa-binder-access:big-binder-team for authorization. Please send us an email at binder-requests@2i2c.org to be added to the GitHub Team.

Acknowledgements #

NASA NSPIRES F.15 High Priority Open-Source Science Award NNH22ZDA001N-HPOSS
The NASA ScienceCore community (originally seeded by TOPS-T award 80NSSC23K0861), an anchor user community whose training delivery requirements helped shape the design of these ephemeral binder services

US Greenhouse Gas Center supports summer school at CIRA

Thu, 20 Jun 2024 00:00:00 +0000

Summer school for inverse modeling of greenhouse gases 2024

The Cooperative Institute for Research in the Atmosphere ( CIRA) is an interdisciplinary cooperation between NOAA research scientists and Colorado State University. CIRA is hosting a summer school for inverse modeling of greenhouse gases using atmospheric data assimilation techniques. The US Greenhouse Gas Center is supporting the workshop by providing 40+ attendees access to their interactive computing hub operated by 2i2c (see our blog post about their launch).

Hub administrators have built a customized software environment with container technology for use at the workshop. In doing so, this bypasses the need for participants to individually install software on their own machines and the online hub provides a consistent and reproducible interactive computing environment that is easily accessible and scalable.

Acknowledgements #

2i2c would like to credit the following individuals for their great efforts supporting this workshop:

Sanjay Bhangar ( Development seed)
Slesa Adhikari (NASA IMPACT)

Hacking the Project Pythia Cook-off with MyST Markdown

Tue, 18 Jun 2024 00:00:00 +0000

Photo courtesy of Dr Debanjana Das

What is Project Pythia? #

Project Pythia is the education working group for Pangeo, a community platform for Big Data geoscience in which 2i2c operates a cloud hub. The core aim of Project Pythia is to spearhead the creation and curation of community-driven, open-source documentation, in the form of “cookbooks”, to enable the adoption of open, scalable and reproducible workflows for geoscientists.

What did 2i2c do? #

Jenny, James and Angus from the 2i2c team participated in the annual Project Pythia Cook-off 2024, a hackathon where cookbook authors and collaborators can spend dedicated time on creating and maintaining their content using Jupyter Book and deploying their cookbooks with GitHub actions.

2i2c teamed up with the infrastructure breakout group during the hackathon, led by Katelyn FitzGerald ( UCAR) and Kevin Tyle ( University at Albany), and members of the Curvenote team also joined the group.

Day 1 #

2i2c deployed and demonstrated a dedicated BinderHub service for Project Pythia that allowed hackathon participants to “self-serve” images of their software environment, which were specified by including a list of packages in an environment.yml file placed in their GitHub cookbook repository. Participants could then pull the image from a container registry into their 2i2c hub (or indeed, any other JupyterHub server) to share and reproduce their computational environments with ease.

Day 2 #

During the first half of the day, we quickly identified a number of issues that were proving to be a maintenance headache for the Project Pythia infrastructure group:

Configuration files for each cookbook were difficult to update at scale. Project Pythia currently have a gallery of over 30 cookbooks!
Changes to Sphinx-based themes inherited from upstream were prone to breaking custom Project Pythia branding downstream.
Executable content was not able to run on the Project Pythia’s dedicated BinderHub hosted on JetStream2 (operated by NSF).
Cookbooks frequently cross-referenced materials from other cookbooks to build upon pre-existing knowledge, but this was not easy to author and the reader experience was not as smooth as it could be.

Following the announcement that Jupyter Book 2.0 will use MyST last month, Rowan (Curvenote) and Angus (2i2c) delivered a compelling demonstration of the MyST ecosystem centered around modern web-first technologies (JavaScript/TypeScript) that offers improved interactivity and accessibility.

In the second half of the day, we decided to use the hackathon to explore migrating the Pythia cookbooks from using a Sphinx-based to a MyST-based document structure and engine. Within one afternoon, the group migrated four cookbooks to use MyST MD

This moment was palpably exciting! It was evident that MyST MD supported backwards compatible content out of the box, which alleviated fears of sunk cost into existing Sphinx-based cookbooks. The migration workflow was as simple as executing the following commands

conda install mystmd

myst.

Day 3 #

We spent this day tackling support for managing a gallery of Project Pythia cookbooks at scale. See the Executable Books blog post for technical details on how we

Centralized configuration
Prototyped a gallery plugin in Python
Fixed a number of bugs related to integrated computation with Binder and JupyterLite
Embraced the referencing and reuse of content with simple markdown syntax for hover-references.

Day 4 #

Looking to the future, we spent time reflecting on our experiences and discussing the potential, transformative impact MyST MD tooling could have in the hands of the scientific community at large, including the communities served by 2i2c. Knowledge-sharing based on static figures and PDFs would fall obsolete and give way to a dynamic, web-first approach to sharing interactive narratives backed by compute from a Jupyter server.

Throughout the course of the hackathon, the rate of iterated development for both end users of the community cookbook and the developers of the open-source tooling was astounding. For example, we were able to quickly expose small bugs ( e.g. support for HTML video tags) in the MyST MD tooling, which were immediately fixed upstream and released within minutes. The feedback loop that connected the user experience with the software tooling was incredibly synergistic, with immediate impact both upstream and downstream that 2i2c hopes to continue replicating across many facets of their operations.

Beyond the Project Pythia Cook-off, the breakout group will continue conversations around strengthening their community of practice and hopefully advocating for wider adoption of MyST MD amongst the scientific community (say hello to some of our group members at SciPy 2024 in July!).

Acknowledgements #

University at Albany (NSF award 2324302): Led the funding acquisition, helped organize and facilitate the event
UCAR (NSF award 2324303): Led the planning and logistics for the event
Project Pythia for organizing this workshop.
Jupyter Book for providing development and collaboration at this workshop.
2i2c / Code for Science and Society (NSF award 2324304): Provided tailored compute services and on-site support
Curvenote: Contributed engineering cycles to MyST MD development.

Researchers at LEAP-Pangeo investigate overlooked sub-grid air-sea heat flux in climate models

Tue, 21 May 2024 00:00:00 +0000

Figure from the preprint showing large and small scale air-sea fluxes are separated. By Julius Busecke et al., licensed under CC BY 4.0

Julius Busecke et al. of the LEAP-Pangeo¹ hub, have recently published a preprint² that investigates “The Overlooked Sub-Grid Air-Sea Flux in Climate Models” using 2i2c infrastructure.

See Julius’ social media post for a more bite-sized outline of the work done.

Well done all! 🎉

Acknowledgements #

This research was conducted using the Pangeo ecosystem and infrastructure provided by 2i2c.

NSF Science and Technology Center (STC) Learning the Earth with Artificial intelligence and Physics (LEAP) (Award # 2019625) ↩︎
doi.org/10.31223/X5WQ47 ↩︎

Digital public goods for Earth system management: U.S. Greenhouse Gas Center launches

Tue, 05 Dec 2023 00:00:00 +0000

Abstract

The International Interactive Computing Collaboration ( 2i2c.org), working with NASA VEDA, Development Seed and other partners, operates an interactive computing platform for The U.S. Greenhouse Gas Center. The U.S. GHG Center, announced yesterday at the 28th annual United Nations Climate Conference (COP-28) in Dubai, is an interagency collaboration of the Environmental Protection Agency (EPA), the National Aeronautics and Space Administration (NASA), the National Institute of Standards and Technology (NIST), and the National Ocean and Atmospheric Administration (NOAA). This note places the launch of the U.S. GHG Center in a scientific, international, and national context and argues that similar digital public goods are needed for humanity to understand and manage the Earth system.

Scientific Context #

It was controversial in 1827 when Joseph Fourier (the discoverer of the law of heat conduction) argued ¹ that the atmosphere keeps the Earth warm, like a puffy down comforter, but it’s not now. Gases in the atmosphere trap heat near Earth. How much heat is trapped depends on the gas mixture. Putting more heat-trapping gases in is like putting a wool blanket on top of the down comforter. Human activity since industrialization is injecting lots more heat-trapping gas into the atmosphere and changing the Earth’s climate.

The science is clear. The up-to-date consensus view of the global scientific community is expressed in the Sixth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC):

International Context #

The international community officially recognized human-influenced climate change at the World Climate Conference (WCC-1) ² in 1979. The 1979 declarationis remarkably prescient and detailed. A complex and interconnected collection of scientific and diplomatic activities were catalyzed by WCC-1. Some important milestones from this history are captured in the chart and numbered list below.

gantt dateFormat YYYY-MM-DD title International Climate Change Milestones section Study WCC-1 (1979) :crit, done, admin0, 1979-02-12, 1979-02-23 WCP :crit, adminA, 1979-06-01, 2025-12-31 CMIP1: crit, done, adminT, 1995-01-01,1995-12-31 CMIP2: crit, done, adminR, 1997-01-01, 1998-12-31 CMIP2+: crit,done,adminS, 2000-05-09, 2001-12-31 CMIP3 :crit, done, adminP, 2004-10-01, 2006-12-31 CMIP5 :crit, done, adminO, 2008-09-01, 2013-03-15 CMIP6 :crit, done, adminQ, 2014-02-01, 2024-12-31 IPCC :crit, admin1, 1988-12-06, 2025-12-31 IPCC-AR1 :crit, done, adminH, 1990-08-01, 1992-06-30 IPCC-AR2 :crit, done, adminI, 1995-01-01,1995-12-31 IPCC-AR3 :crit,done, adminJ, 2001-01-01, 2001-12-31 IPCC-AR4 :crit, done,adminK, 2007-01-01,2007-12-31 IPCC awarded Nobel Prize :crit, done, adminN, 2007-10-12, 2007-11-12 IPCC-AR5 :crit,done,adminL, 2014-01-01,2014-12-31 IPCC-AR6 :crit,done,adminM,2023-01-01,2023-12-31 section Treaties Rio Earth Summit (1992) :crit, done, adminC, 1992-06-03, 1992-06-14 UNFCC :crit, admin2, 1994-03-21, 2025-12-31 Berlin (COP-1) :crit, done, adminE, 1995-03-28, 1995-04-07 Byrd-Hagel Resolution :crit, done, adminX, 1997-07-25, 1997-07-30 Kyoto (COP-3) :crit, done, adminD, 1997-12-01, 1997-12-10 Kyoto Protocol :crit, done, admin3, 1997-12-11, 2020-12-31 Paris (COP-21) :crit, done, adminF, 2015-11-30, 2015-12-12 Paris Agreement :crit, adminG, 2016-11-04, 2025-12-31 Glasgow (COP-26) :crit, done, adminV, 2021-10-31, 2021-11-12 Dubai (COP-28) :crit, adminW, 2023-11-20, 2023-12-12 ETF :crit, adminU, 2024-01-01, 2025-12-31

The table above describes a subset (for a more systematic review see ³, ⁴) of key milestones in global efforts to understand and address climate change. A glossary of acronyms and additional background:

The First World Climate Conference (WCC-1) ² was held in 1979.
The World Climate Programme ( WCP), an activity overseen by the World Meteorological Organization was established after WCC-1. WCP, in partnership with other organizations, operates programs (e.g. the World Climate Research Program (WCRP)) that organize and integrate international scientific efforts to understand the climate. The WMO also operates the Integrated Global Greenhouse Gas Information System (IG3IS), a natural partner for the emerging work described below.
WCRP manages the Common Model Intercomparison Project (CMIP). CMIP serves as a kind of league for intercomparing models of the Earth’s climate system developed by teams who approach the problems with different methods and designs. Intercomparison, an approach that enables finding the consensus views of teams with divergent approaches to problems, is used in other modeling scenarios.
Research papers on the climate are rapidly produced by scholars from essentially all knowledge disciplines. This overwhelming stream of content, like snowflakes in a blizzard, is coalesced into coherent and carefully scrutinized IPCC Assessment Reports by the Intergovernmental Panel on Climate Change (IPCC).
The Earth Summit held in Rio de Janeiro in 1992 led to the United Nations Framework Convention on Climate Change (UNFCC). The UNFCCC is an international treaty that recognizes the dangers to the climate system caused by human activity, calls for ongoing study, and establishes recurring Conference of the Parties (COP) meetings.
COP-3 (Kyoto) led to the Kyoto Protocol Treaty.
The Byrd-Hagel Resolution was a unanimous United States Senate Resolution that stipulated the United States would not sign treaties promising greenhouse gas reductions by developed countries that did not mandate similar reductions from developing countries. This killed USA participation in the Kyoto Protocol Treaty.
The Paris Agreement, established at COP-21 (Paris), effectively replaces the Kyoto Protocol, includes specifications by participant countries on greenhouse gas reductions called National Determined Contributions (NDCs). The United States entered the Paris Agreement under President Obama, exited under President Trump and rejoined under President Biden.
COP-26 (Glasgow)established an accountability system for the Paris Agreement called the Enhanced Transparency Framework (ETF). Participant countries to the Paris Agreement will submit their first Biennial Transparency Reports (BTR1) under the ETF in 2024.

🇺🇸 U.S.A. Context #

The United States Environmental Protection Agency (EPA) annually releases the Inventory of U.S. Greenhouse Gas Emissions and Sinks reports. These reports are submitted to the United Nations in accordance with the UNFCCC. The EPA openly shares software, tools, data, and builds capacity to assist other nations to assemble their own greenhouse gas inventories.

An Interagency Working Group (IWG) on the Social Cost of Carbon was set up by the Obama Administration in 2010. The IWG, renamed as the Interagency Working Group on Social Cost of Greenhouse Gases in 2016, synthesizes research on integrated assessment modelling to quantify the dollar costs associated to damage caused by an incremental increase in GHG emissions in a given year. Quantifying the impacts of GHG emissions in monetary terms is vital to effective rulemaking across the Federal Government. This EPA fact sheet on the social costs of carbon provides further background. A 2017 consensus report ⁵ of the National Academies of Science Engineering and Medicine (NASEM) offered recommendations for ongoing research to improve the assignment of social costs to GHG emissions.

Other federal agencies have developed expertise, data, and analyses that give insights into GHG emissions that compliment and potentially extend the Inventory reports developed annually by the EPA. How should the United States integrate federal agency efforts to monitor and measure greenhouse gas emissions? A 2022 NASEM consensus report ⁶ investigated this question. In January of this year, the Biden Administration released guidance⁷ for federal agencies on incorporating GHG emissions information in policies and reports. Shortly thereafter, a draft federal strategy to advance an integrated greenhouse gas monitoring system was released by NASA with a request for public input.

Some important insights from the NASEM consensus report, the draft federal strategy, and the IWG reports:

Data streams on greenhouse gas emissions can be sorted into two broad categories:
- Activity-based: Activity-based data, sometimes called “bottom-up” data, quantify GHG emissions by measuring activities that generate GHG emissions. Economic or business activity data (gallons of diesel sold in a county on Tuesday; miles flown by a 747 in October) can be converted into quantified GHG emissions information.
- Atmospheric-based: Atmosphere-based data, sometimes called “top-down” data, quantify GHG emissions by performing atmospheric measurements. For example, the Orbiting Carbon Observatory (OCO2) remotely senses $CO_2$ from space.
- A hybrid approach that blends activity-based and atmospheric-based GHG data has the potential to provide new insights.
Interoperable and easily accessed tools and data products for analyzing GHG emissions information and assigning costs should be made available across the Federal government.
GHG emissions information is needed in scenarios outside the international context of participating nations reporting for UNFCCC and Paris Agreement ETF compliance:
- Subnational governments – cities, states, provinces, counties – want GHG data products to measure their progress on emission reduction programs.
- Facilities – harbours, toll roads, power plants, factories, universities – similarly want GHG data products.
- Companies – airlines, trucking, construction – want GHG data products.
Reliable and transparent GHG emissions information is required to enable effective environmental-social-governance (ESG) investment without “greenwashing”.
Accurate and improving quantifications of the social costs associated to GHG emissions require ongoing research.

The U.S. Greenhouse Gas Center #

The launch of U.S. GHG Center, an interagency collaboration of the Environmental Protection Agency (EPA), the National Aeronautics and Space Administration (NASA), the National Institute of Standards and Technology (NIST), and the National Ocean and Atmospheric Administration (NOAA), was announced on 2023-12-04 at COP-28 (Dubai)

How much GHG emission is generated through oil and gas production? How much GHG emission is generated by urban centers? Do landfills contribute significant GHG emissions? How do human-generated GHG emissions compare to natural sources of GHG emissions? The U.S. GHG Center is designed to assemble the data, tools, and people to scientifically address these kinds of questions.

The U.S. Greenhouse Gas Center builds on EPA’s leadership to openly share the data and tools for the Inventoryand the 2023 Year of Open Science. Instead of building a walled garden with proprietary technology from a vendor selected through RFP, the Center launched a prototype platform using curated open source tools integrated with public federal data. This open toolchain approach amplifies the open data efforts developed over the past two decades.

The U.S. GHG Center’s interactive computing platform is open source science infrastructure. The platform is:

transparently operated by 2i2c on a cloud data center under a right to replicate that ensures zero vendor lock-in with data integrations and visualizations built with using software from a vibrant open source ecosystem by Development Seed, NASA VEDA, and collaborators;
proximate to and optimized ⁸ for analyzing geospatial data (e.g. NASA, NOAA);
designed to be a digital public good.

Research, data and recommendations developed by scientists and engineers that influence policies set by democratic governments should be accessible by voters. No entity should own the ways humans communicate and learn about the Earth system. The U.S. Greenhouse Gas Center’s generous approach to digital infrastructure – an open toolchain adjacent to open data – is vital for democracy and should be replicated across government agencies worldwide.

Acknowledgements #

This work was developed in collaboration with Development Seed and NASA VEDA, key partners in building the U.S. Greenhouse Gas Center’s open source science infrastructure.

References #

Mémoire sur les températures du globe terrestre et des espaces planétaires. in Oeuvres de Fourier: Publiées par les soins de Gaston Darboux (eds. Fourier, J. B. J. & Darboux, J. G.) vol. 2 95–126 (Cambridge University Press, 2013). ↩︎
World Climate Conference. World Climate Conference - Declaration and supporting documents. library.wmo.int/records/item/54699-world-climate-conference-declaration-and-supporting-documents (1979). ↩︎ ↩︎
Gupta, J. A history of international climate change policy. WIREs Climate Change 1, 636–653 (2010). ↩︎
Zillman, J. A history of climate activities. WMO Bulletin 58, (2009). ↩︎
Valuing Climate Changes: Updating Estimation of the Social Cost of Carbon Dioxide. (National Academies Press, 2017). doi: 10.17226/24651. ↩︎
Greenhouse Gas Emissions Information for Decision Making: A Framework Going Forward. (National Academies Press, 2022). doi: 10.17226/26641. ↩︎
CEQguidance. National Environmental Policy Act Guidance on Consideration of Greenhouse Gas Emissions and Climate Change. Federal Register www.federalregister.gov/documents/2023/01/09/2023-00158/national-environmental-policy-act-guidance-on-consideration-of-greenhouse-gas-emissions-and-climate (2023). ↩︎
Abernathey, R. P. et al. Cloud-Native Repositories for Big Scientific Data. Computing in Science & Engineering 23, 26–35 (2021). ↩︎

A QGIS desktop in the cloud with JupyterHub

Sat, 05 Aug 2023 00:00:00 +0000

The QGreenland Researcher Workshop

JupyterHub is a versatile platform that can serve a desktop with Geospatial Information Systems (GIS) software in the cloud. This was demonstrated by the QGreenland Researcher Workshop that was hosted by the NASA CryoCloud hub. The hands-on workshop trained 25-30 researchers, from Germany, India, France, Canada, Poland and the United States, on how to work with geospatial data in an open science framework.

QGreenland Overview #

QGreenland is an open-source geospatial data package designed for QGIS, a community-owned GIS platform. It focuses on Greenland, offering researchers and educators a comprehensive toolset for FAIR (findable, accessible, interoperable and reproducible) data analysis. The package integrates a variety of datasets into a single, easy-to-use data-viewing and analysis platform, supporting both offline and online use. This makes it particularly valuable for remote fieldwork and areas with limited internet access.

Workshop Success #

The QGreenland workshop demonstrated several key benefits of using JupyterHub for cloud-based GIS:

Accessibility: Participants from across the world could access the same powerful GIS tools through a web browser, eliminating the need for complex local installations while enhancing reproducibility
Cloud block storage: Using a JupyterHub in the cloud allowed for faster data access than a traditional NFS file store by provisioning each user with an elastic block store disk, reducing load times from 5 minutes to under 3 seconds.
Cost Efficiency: Utilizing the CryoCloud JupyterHub instance managed by 2i2c drastically cut down setup costs and time, with only minimal cloud operating expenses of roughly $1/person/day.

Conclusion #

The success of the QGreenland workshop underscores the potential of integrating interactive software applications in JupyterHub. This approach not only democratizes access to advanced geospatial tools but also fosters a collaborative research environment. We look forward to supporting more workshops for QGreenland in the future!

Want to know more? Check out the companion post by QGreenland on the Jupyter Blog

Acknowledgements #

Trey Stafford (CIRES)
Matthew Fisher (CIRES)
*Fisher, M., *T. Stafford, T. Moon, and A. Thurber (2023). QGreenland (v3) [software], National Snow and Ice Data Center.
Snow, Tasha, Millstein, Joanna, Scheick, Jessica, Sauthoff, Wilson, Leong, Wei Ji, Colliander, James, Pérez, Fernando, James Munroe, Felikson, Denis, Sutterley, Tyler, & Siegfried, Matthew. (2023). CryoCloud JupyterBook (2023.01.26). Zenodo. 10.5281/zenodo.7576602

* Denotes co-equal lead authorship

The why, what, and how of our NASA Openscapes cloud infrastructure: 2i2c JupyterHub and corn environment

Thu, 17 Nov 2022 00:00:00 +0000

We recently shared a demo of our infrastructure stack with the OpenScapes community. Check out the blog post about it here.

Reflections on the Jack Eddy Symposium

Thu, 14 Jul 2022 00:00:00 +0000

Reflections on the Jack Eddy Symposium #

2i2c supported and participated in the 3rd Eddy Cross Disciplinary Symposium held recently in Vail Colorado. The event was hosted by the Cooperative Programs for the Advancement of Earth System Science ( CPAESS) team at the University Corporation for Atmospheric Research (UCAR) with support from NASA.

Context #

The Symposium was framed by the interesting and interdisciplinary scientific career of the late Jack Eddy. Eddy’s legacy was highlighted and his influence has been extended by dynamic leadership from NASA Program Officer Madhulika Guhathakurta (Lika). Lika helped launch and has sustained NASA’s Living with a Star (LWS) program over the past two decades. Prior to LWS, NASA had a variety of siloed efforts focused on near-Sun and near-Earth behavior. The LWS program led to an integration of these efforts under “system science” or “systems engineering” approaches and an expressed desired to connect LWS research activities with impacts on Earth (society, biology, culture, etc.). The program has expanded to include explorations of similar questions arising around other planets in our solar system and the recently discovered collection of exoplanets. Scientists from diverse disciplines (plasma physics, stellar evolution, atmospheric chemistry, space weather, planetary science,…) work together on “cross disciplinary” research that helps us understand our lives near our star.

The Symposium focused on three disciplinary areas (Exoplanets; Sun-Climate and Star-Climate interactions; Risk and resilience of space weather) unified under the cross-cutting thread of open science. Frequent references were made to the upcoming 2023 Year of Open Science and NASA’s Transform to Open Science (TOPS) mission. Symposium attendees listened to talks surveying the four areas in the morning and participated in hackathon-style breakout projects during the afternoons. Work on the projects launched at the Eddy Symposium continues. The space weather group is investigating ways to make the power grid more resilient. The Sun-climate group is exploring plans to establish an institute focused on Sun-climate interactions and improve connections between climate and heliophysics research communities. The exoplanets team is developing tools to programmatically compare Sun-Earth and star-exoplanet interactions.

2i2c’s role #

2i2c, with input from Symposium CoChair’s Dan Marsh and Ryan McGranaghan, rapidly deployed a cloud-hosted JupyterHub for use during the event. The hub provided a shared space for participants to explore data, run analyses, and collaborate with one another using modern tools including Zarr, Xarray and Dask Gateway. Access to the interactive computing platform was granted to any member of the Symposium’s GitHub organization. The work to set up that hub, openly chronicled in this GitHub issue ( 2i2c-org/infrastructure#1329 ), included swapping out a Pangeo-style software environment for a heliophysics-specific resource developed by HelioCloud with special thanks to Brian Thomas!

2i2c co-founder Fernando Pérez gave a talk on how he is “living la vida nube”. Fernando described the ways he, research collaborators, and students are using the Jupyter ecosystem. Diverse and curated tools in Jupyter hubs for the Jupyter Meets the Earth Project and Berkeley’s data science programs were highlighted. The talk showcased how these tools have been integrated to support individuals and communities of practice in data-driven research. In response to requests from the organizers and participants, Fernando gave a demonstration on how to use the hub 2i2c set up for the Symposium and an introduction to version control using git.

2i2c co-founder Jim Colliander gave a talk titled Governing the Science Commons. Three key points from Jim’s talk were: the virtue that should guide the improvements to the scientific enterprise is intellectual generosity; implementation of intellectual generosity into science requires commons-based governance; the convergence of open source tools that support data-intensive collaborative research and learning (as showcased by Fernando) and agency interest ( NASA TOPS, UNESCO) in open science is an inflection point for global change. The talk ended with a call to action for the diverse communities represented at the Symposium to improve the ways we do science.

Things we learned #

Our experience with the Symposium taught 2i2c a few things.

We learned that our engineering team can rapidly deploy interactive computing resources to support a research and education community. Along the way, we confirmed what we’ve been learning from Pangeo and the neuroscience communities: flexible methods to customize the software environment are necessary. We confirmed that our developing shared responsibility model, enabling domain-specific experts to provide curated toolchains for their communities while leveraging 2i2c’s infrastructure expertise, is the right approach.

We learned that managing access to the hub using members in a GitHub organization works but involves some toil since organizers had to work through the GitHub invitation process for each participant. We are exploring others ways to systematically grant event participants access to a hub.

The Eddy Symposium experience reminded us that some skill at using git is vital for collaboration. Fernando’s brief introduction to git provided the mental model of working on a directed acyclyic graph. Roving support from Jim and Fernando during the afternoon hackathon sessions normalized the skill level across the groups but not without a few merge conflicts and frustration. Near the beginning of future workshops, we will follow an improved approach aimed at establishing good git hygeine and familiarity with the collaboration workflow using a simple pull request.

We are honored to have had the opportunity to collaborate with the Eddy Symposium team by managing infrastructure and participating in the workshop. We learned a lot about how to support the Helio and adjacent communities, and are proud of the ways in which our infrastructure helped them do their work more openly and effectively. We hope to build on these successes with these and other communities in the future!

Pangeo Cloud goes live on 2i2c!

Tue, 16 Nov 2021 00:00:00 +0000

Pangeo Cloud is an experimental service providing public cloud-based data-science environments for data-intensive geoscience research. We have recently finished re-creating the Pangeo community JupyterHub hosted on GCP in the 2i2c-org/infrastructure repository. This is a huge milestone in our partnership with Pangeo to provide expertise and operations of cloud-based, vendor-agnostic Jupyter infrastructure and workflows.

For users of Pangeo Cloud, the switch should have been a smooth one! The new hub should behave nearly identically to the old one, and will be managed by 2i2c engineers moving forward, in partnership with the Pangeo community. It will be available at the same URL ( us-central1-b.gcp.pangeo.io) and there’s no need to worry about your home directories, they were synced to the new hub only a few days before the migration took place. Development and operations on this hub will all be done in the open and we invite participation and feedback from others in our infrastructure work. Please see this Discourse thread as an initial place to provide feedback.

On 22nd November 2021, the old Pangeo GCP JupyterHub will be shut down, and the project will move forward on the new 2i2c Pangeo Hub. Moving forward, we plan to collaborate together in order to find new pathways for development in the Jupyter ecosystem - we will share more ideas of things we will work on soon!

History of Pangeo Cloud Hubs #

Pangeo has pioneered a new model in using open source and cloud-agnostic infrastructure to support scientific research in the cloud.

The first Pangeo cloud JupyterHub (pangeo.pydata.org; now defuct) was deployed for the 2017 American Meteoroligical Society Meeting; since then, the Pangeo community has iterated through several different versions of prototype cloud-based hubs. This allowed for many new workflows that enabled a more open and collaborative pathway to doing world class research, and included access to datasets and computational resources that were previously unattainable. Pangeo achieved this by working in partnership with open source communities and building technology that leveraged modular open source components for their platform.

In the last several years, Pangeo have built a thriving community of practice around this infrastructure. However as the community has grown, so has the need for more reliable and dedicated operational and developmental support since parts of the Pangeo stack require dedicated expertise and attention to managed. Modern scalable cloud infrastructure is one example of this. Maintaining a complex JupyterHub with many users is a difficult task, and has required significant resources from the Pangeo Project up to this point.

The Pangeo-2i2c Partnership #

2i2c is a non-profit team that develops and operates cloud infrastructure for interactive computing workflows. We have extensive experience in Jupyter workflows in the cloud and a long history of contributions to projects in this ecosystem. We have built a cloud deployment management system that allows us to centralise and configure the deployment of many independent JupyterHubs, empowering communities to leverage the same infrastructure (and team!) for JupyterHubs running in the cloud.

Similarly to Pangeo, all of 2i2c’s core infrastructure is cloud- and vendor-agnostic, and follows a model of building open source tools and giving back to those communities. Our partnership with Pangeo began through 2i2c’s core competency in these areas and the similarity between the two project’s technical stacks.

We’ve begun a partnership whereby 2i2c will manage Pangeo’s cloud infrastructure and lead efforts to develop new features, in partnership with open source communities. We sketched out a few ideas to focus on in this kick-off thread on Discourse. This approach allows each community to focus on it’s core strengths: Pangeo will continue to grow an open community and scientific software ecosystem around geospatial analytics, and 2i2c will oversee the development and operations of the core cloud infrastructure stack that powers Pangeo’s workflows. In some areas we are still experimenting with different collaboration models to ensure that the needs of the Pangeo community are met in a way that is also sustainable for 2i2c. Over the coming weeks, you may see some conversations (and threads for feedback!) about different support and operations models that work best for the community. We are excited to use this as an opportunity to learn more about how to serve more complex and diverse communities like Pangeo.

Acknowledgements #

We are extremely grateful to the Pangeo project for giving us the opportunity to serve their community, as well as the Moore Foundation for funding this work. We look forward to a long partnership ahead! 🚀

Earth-Science | 2i2c

NASA Open Science ScienceCore tutorial available at github.com/sciencecore

Acknowledgements #

Supporting NASA Openscapes Champions with Cloud Infrastructure

Communities learning from one another - Project Pythia and ICESat-2 Hackweeks

Learn more about these communities #

Harnessing Marine Open Data Science for Ocean Sustainability in Africa, South Asia and Latin America

Open infrastructure for collaborative geoscience with Project Pythia: Learning how to deploy a BinderHub on Jetstream2

Project Pythia and the “Jupyter notebook obsolescence” problem #

Leveraging NSF-funded cyberinfrastructure for BinderHub #

Learning how to deploy on OpenStack #

Acknowledgements #

NASA VEDA & 2i2c Update for Q4 2024 (Oct-Dec 2024)

Automated backups and alerting with jupyterhub-home-nfs #

Enable users to dynamically build environments with jupyterhub-fancy-profiles #

“Open in QGIS” from VEDA UI #

Other updates #

Acknowledgements #

2i2c communities at AGU 2024

ED31G-2272 Breaking down the barriers to Open Science with Project Pythia #

Abstract #

Authors #

ED31G-2277 PACE Hackweek: An open community keeping up with PACE #

Abstract #

Authors #

IN13A-2147 Including more solutions and more solvers via actionable open science #

Abstract #

Authors #

IN34A-01 Beyond Open Data: Ensuring True Accessibility for All (Invited) #

Abstract #

Authors #

Introducing GeoLab - An EarthScope JupyterHub for Enabling Collaborative Cloud-Native Geophysical Data Analysis and Skill Development Workshops #

Abstract #

Authors #

U13A-2349 Sharing recipes for cloud computing: the Project Pythia Cookbook Initiative #

Abstract #

Authors #

U13A-2350 Supporting NASA Earthdata users in the Cloud: NASA Openscapes JupyterHub and User Onboarding & Fledging #

Abstract #

Authors #

V31A-08 VICTOR – A new Cyber-infrastructure for Volcanology #

Abstract #

Plain-language Summary #

Authors #

From the GeoLab team - Pancakes are the future of geophysical data processing

Keeping PACE with GPU enabled compute to detect global cloud cover using satellite data

Machine Learning with GPUs #

Managing shared memory on 2i2c hubs #

Conclusion #

References and Acknowledgments #

Ephemeral Interactive Computing for the AmeriGEO Workshop

Acknowledgements #

Collaborating with Development Seed to deliver cyberinfrastructure for NASA VEDA

Better image management and testing #

Automatically pulling example notebooks on startup #

Opening specific visualizations in QGIS via URL #

Better Profile Selection #

Supporting workshops #

Ongoing Collaboration #

Acknowledgements #

Openscapes Host a Surface Biology and Geology Workshop with Shared Password Feature

Acknowledgements #

Determining Climate Risks with NASA Earthdata Cloud at Scipy 2024

Acknowledgements #

Ephemeral Interactive Computing for NASA Communities

What did we do? #

How did we do it? #

What next? #

Can I use this ephemeral hub service? #

Acknowledgements #

US Greenhouse Gas Center supports summer school at CIRA

Acknowledgements #

Hacking the Project Pythia Cook-off with MyST Markdown

What is Project Pythia? #

What did 2i2c do? #

Day 1 #

Day 2 #

Day 3 #

Day 4 #

Acknowledgements #

Automated backups and alerting with `jupyterhub-home-nfs` #

Enable users to dynamically build environments with `jupyterhub-fancy-profiles` #