Notification Bot (Jun 27 2022 at 15:41): Notification Bot (Jun 27 2022 at 15:41):Public stream created by Julia Kent. Description:
Coordination for Pythia Summer 2022 Hackathon
Notification Bot (Jun 27 2022 at 15:43):Julia Kent has marked this topic as resolved.
Notification Bot (Jun 27 2022 at 15:43):Julia Kent has marked this topic as unresolved.
Julia Kent (Jun 27 2022 at 15:44):Post here for coordination for Pythia summer 2022 hackathon
Kevin Tyle (Jun 27 2022 at 15:47):@Max Grover could you please place the link to the Pangeo binder hub below?
Max Grover (Jun 27 2022 at 15:47):You betcha, here it is:
Pangeo Hub Link - https://binder-staging.2i2c.cloud
Julia Kent (Jun 27 2022 at 16:08):20220627_083946.jpg 20220627_084031.jpg
Look who showed up for the hackathon!
Julia Kent (Jun 27 2022 at 16:21):Link to the welcome document
Link to the agenda
Max Grover (Jun 27 2022 at 17:02):Link to Brian's breakout room - https://meet.google.com/dec-mtfo-wdh
Heather Craker (Jun 27 2022 at 20:19):Hi all! I missed the planning session this morning. Has a standard been decided regarding what data we can use for the cookbooks and how it should be included? Or are we able to use any public data and load it in using any of the "usual" methods (i.e. xarray, pandas)
Brian Rose (Jun 27 2022 at 20:23):Any public data is fair game at this point.
Heather Craker (Jun 27 2022 at 20:27):Sounds good. I might use some from the geocat datafiles repository since it's extremely easy to access
Kevin Tyle (Jun 27 2022 at 20:47):woo-hoo, tried my initial prototype HRRR/AWS notebook with the new 2i2c binderhub and it worked! Looks like it is using Python 3.7, though, when it creates the conda environment.
Brian Rose (Jun 27 2022 at 21:04):That's terrific @Kevin Tyle. What happens if you specify a different Python version in your environment.yml?
Kevin Tyle (Jun 27 2022 at 21:20):@Brian Rose Currently I don't specify a Python version, and have gotten different results depending on what host I build the environment on (3.10 on my local build, 3.7 on binder)
Brian Rose (Jun 27 2022 at 21:55):All, let's reconvene in "plenary" at 4 pm in the main conference room for a bit of a debrief on today's hacking!
Julia Kent (Jun 27 2022 at 21:57):I updated the conf.pys to point to the correct cookbook gallery location (in the new domain) across all of the cookbooks and the template. Sorry I did not do that before we started the hackathon -- You may need to do a git pull
John Clyne (Jun 27 2022 at 21:58)::+1:
Julia Kent (Jun 28 2022 at 03:14):Anyone who wants to hike from NCAR to the bat cave tomorrow bring good shoes, hiking clothes for the warm weather, and a water bottle! The trail is about 3.5 miles, 800 ft elevation gain, and will take us about 1-2 hours. We don't have to do the whole thing, but this is the closest "destination" hike from the NCAR trail head.
https://www.alltrails.com/trail/us/colorado/mallory-cave
Julia Kent (Jun 28 2022 at 14:56):Running just a few minutes late!
Orhan Eroglu (Jun 28 2022 at 15:16):Can someone put the link to this main room meeting?
Kevin Paul (Jun 28 2022 at 15:16):Do you have the calendar invite?
Kevin Paul (Jun 28 2022 at 15:17):It's the Google Meet link in the calendar event:
Pythia Hybrid Hackathon
Tuesday, June 28 · 8:30am – 5:00pm
Google Meet joining info
Video call link: https://meet.google.com/hiz-ubpv-jgf
Or dial: (US) +1 401-542-2486 PIN: 590 014 902#
More phone numbers: https://tel.meet/hiz-ubpv-jgf?pin=4741406198038
Orhan Eroglu (Jun 28 2022 at 15:22):No, I didn't; thanks!
Julia Kent (Jun 28 2022 at 15:30):Cookbook Contributor's Guide --> https://github.com/ProjectPythiaTutorials/cookbook-template/blob/main/CONTRIBUTING.md
Julia Kent (Jun 28 2022 at 15:35):@Kevin Paul I don't see an "Issues" tab in the new repository? https://github.com/ProjectPythiaCookbooks/projectpythiatutorials.github.io
Also should we rename it and the domain to projectpythiacookbooks.github.io ?
Kevin Paul (Jun 28 2022 at 15:35):Yes. I'm still in the middle of doing a lot of things. It's not done yet. I'll let you know.
Max Grover (Jun 28 2022 at 15:45):Here are the instructions on what needs to be updated in each cookbook to run properly on the pangeo binder + move the jupyterbook build files to the root directory https://github.com/ProjectPythiaTutorials/cookbook-template/issues/28#issuecomment-1168881585
Kevin Paul (Jun 28 2022 at 15:57):IMPORTANT: The cookbooks organization has now been created. I believe it is ready to start moving repos over to it. Please look for an invitation to the new organization and accept it, if you haven't. Then you can start transferring your repos over the the new organization.
NOTE: If you want to transfer a repo to the new org that you don't want to move but copy do this by making a fork. You can detach a fork from its upstream origin later.
Kevin Paul (Jun 28 2022 at 16:01):I have moved the cookbook-template repo to the new org.
Heather Craker (Jun 28 2022 at 16:12):@Kevin Paul I don't see an invite. Can you add me to the new org?
Kevin Paul (Jun 28 2022 at 16:12):Heather, were you not invited to the ProjectPythiaTutorials org, either?
Anissa Zacharias (Jun 28 2022 at 16:13):@Kevin Paul I also never got invites for either org
Heather Craker (Jun 28 2022 at 16:14):No I don't believe so. I'm a member of ProjectPythia as a whole, but I'm not part of any of the sub orgs
Kevin Paul (Jun 28 2022 at 16:14):Ah. That's why I didn't invite either of you. I'll invite you to both the tutorials org and the cookbooks org.
Brian Rose (Jun 28 2022 at 16:25):Kevin Paul said:
IMPORTANT: The cookbooks organization has now been created. I believe it is ready to start moving repos over to it. Please look for an invitation to the new organization and accept it, if you haven't. Then you can start transferring your repos over the the new organization.
NOTE: If you want to transfer a repo to the new org that you don't want to move but copy do this by making a fork. You can detach a fork from its upstream origin later.
What is the advantage of forking rather than moving? Is that what you recommend for migrating individual Cookbook repos?
Kevin Paul (Jun 28 2022 at 16:28):If you fork it, you get a "copy" of the repo in both orgs. So, for example, we probably want a main organization site (tutorials.projectpythia.org and cookbooks.projectpythia.org) for both organizations. And the content for both of those repos will look very similar, with very minor differences. By copying and changing, you make the job of setting up both main sites much easier.
Kevin Paul (Jun 28 2022 at 16:28):If it doesn't make sense to have a copy in both orgs, then just transfer it. (e.g., like the cookbook-template repo).
Kevin Paul (Jun 28 2022 at 16:29):GitHub support has an automated mechanism to request that a fork be "detached" from its origin. So, when the two repos start to diverge, we can cleanly separate them and evolve them independently.
Kevin Paul (Jun 28 2022 at 16:46):@Julia Kent! I said I would take care of the tutorials org site!
Julia Kent (Jun 28 2022 at 16:47):Sorry I just drafted 2 PRs. Feel free to ignore
Kevin Paul (Jun 28 2022 at 16:47):I've already made the changes.
Kevin Paul (Jun 28 2022 at 16:47):I've closed the PRs.
John Clyne (Jun 28 2022 at 16:49):@Kevin Paul is it ok to open new cookbook content issues here? Thanks
Kevin Paul (Jun 28 2022 at 16:50):Yes. It should be okay.
Max Grover (Jun 28 2022 at 16:58):@Julia Kent - just submitted a PR updating the link to the radar cookbook's new home, and a small tag change
John Clyne (Jun 28 2022 at 17:16):Hi All,
Just a couple of quick updates for everyone.
For those working on migrating the Pangeo Physical Oceanography notebooks, it does not look like we're going to resolve the data access issues in the next couple of days. We're going to have to reach out to Ryan and explore options. So, please just do what you can to address cosmetic issues and we'll table the PRs for now.
There are two new issues under the newly created ProjectPythiaCookbooks org (thanks Kevin!) for migrating Pangeo notebooks that hopefully will not present data access issues, and would be super useful to have in the Pythia collection: CESM LENS on AWS and "tutorials". It would be awesome if somebody(s) would like to take these on!
Drew Camron (Jun 28 2022 at 17:18):Do we want to open github discussions on the cookbooks org or any of the new org's repos themselves? I don't want to further pollute the waters re finding discussions, but they could be valuable spaces once the moving and shaking settles down a bit
Max Grover (Jun 28 2022 at 17:20):I'd be happy to help with the LENS on AWS one (and add in the CESM2-LE one to that) since I helped with those notebooks last fall...
Kevin Paul (Jun 28 2022 at 17:23):Hey, all! Something has come up and I need to take care of my daughter this afternoon.
John Clyne (Jun 28 2022 at 17:24):Max Grover said:
I'd be happy to help with the LENS on AWS one (and add in the CESM2-LE one to that) since I helped with those notebooks last fall...
Thanks, Max!
John Clyne (Jun 28 2022 at 17:29):Drew Camron said:
Do we want to open github discussions on the cookbooks org or any of the new org's repos themselves? I don't want to further pollute the waters re finding discussions, but they could be valuable spaces once the moving and shaking settles down a bit
I'm wondering whether it makes sense to have discussions enabled anywhere other than the main (site)[https://github.com/ProjectPythia/projectpythia.github.io/]. @Drew Camron you may some good points here. Personally, I have trouble keeping up with information spread out across multiple different repos. I'm guessing it can be a challenge for the user community as well. As you pointed our earlier, Drew, I really like what Fatiando did with their contact us page to help guide uses to the right communication channels.
Drew Camron (Jun 28 2022 at 17:45):Sounds like consolidating the different forums, adding helpful tags, and making a clearer "contact us" page would be the way to go
Drew Camron (Jun 28 2022 at 17:51):I am heading out for the day! I will be around for hopefully the entire session tomorrow and can be around to respond to any asynchronous requests for feedback later today.
John Clyne (Jun 28 2022 at 18:02):@Julia Kent a couple of folks were asking about hiking, but are concerned about the heat. Do you have any thoughts? The group in the penthouse is heading down to the cafeteria.
Julia Kent (Jun 28 2022 at 18:53):I was thinking we'd hike after work. Maybe sync up at 4 again and then hike at 4:30?
Julia Kent (Jun 28 2022 at 18:53):It doesn't look like it will get that much hotter than it already is
Orhan Eroglu (Jun 28 2022 at 20:07):Kevin Paul said:
Ah. That's why I didn't invite either of you. I'll invite you to both the tutorials org and the cookbooks org.
I didn't get invites either
Julia Kent (Jun 28 2022 at 20:50):Can everyone add a thumbnail image to your cookbooks? This thumbnail will be used on your landing page and for the gallery card.
See https://github.com/ProjectPythiaCookbooks/projectpythiacookbooks.github.io/pull/14
I'm hoping to grab the thumbnail from each repository rather than having thumbnails saved in the cookbook gallery repository.
Julia Kent (Jun 28 2022 at 20:51):I think @James Morley and I have the code in place to do this, but won't implement it until https://github.com/ProjectPythiaCookbooks/projectpythiacookbooks.github.io/pull/14 is merged
Kevin Tyle (Jun 28 2022 at 21:25):@Julia Kent where should I place the thumbnail in my cookbook? Does the image need to be of a particular HxW dimension?
Julia Kent (Jun 28 2022 at 21:26):At the top level, I don't believe so but if it looks bad we'll address that.
Brian Rose (Jun 28 2022 at 21:40):Kevin Tyle said:
Julia Kent where should I place the thumbnail in my cookbook? Does the image need to be of a particular HxW dimension?
You can see an example in the Template repo here: https://github.com/ProjectPythiaCookbooks/cookbook-template/pull/44 (although not yet merged in)
Kevin Paul (Jun 29 2022 at 14:00):Hey! My daughter is sick today and I need to watch her. I may be in and out.
Julia Kent (Jun 29 2022 at 14:03):I hope she feels better soon!
Kevin Paul (Jun 29 2022 at 14:04):Thanks! She has almost no symptoms except being tired and has a mild fever. Weird.
Kevin Paul (Jun 29 2022 at 14:05):I’ll work while she sleeps.
Kevin Tyle (Jun 29 2022 at 15:13):Here's a link to the GitHub status page, which includes the current problem with GitHub actions:
https://www.githubstatus.com/incidents/qzmhbwp7f6nn
Julia Kent (Jun 29 2022 at 15:26):GH actions is back up! but degraded
John Clyne (Jun 29 2022 at 15:29):Max reports that GitHub actions is working again. Yay!
Kevin Paul (Jun 29 2022 at 15:59):I have time to look at the Netlify problem now. I'll take a look.
Julia Kent (Jun 29 2022 at 16:07):I'm going to focus on using https://api.github.com/users/Projectpythiacookbooks/repos to grab all the repository names for the cookbooks
Orhan Eroglu (Jun 29 2022 at 16:11):I am having issues with accessing the following catalog:
intake.open_catalog("https://raw.githubusercontent.com/pangeo-data/pangeo-datastore/master/intake-catalogs/ocean.yaml")
Here is the error message I am getting:
ValueError: Bad Request: https://storage.googleapis.com/download/storage/v1/b/pangeo-ecco-eccov4r3/o/eccov4r3%2F.zmetadata?alt=media
User project specified in the request is invalid.
Above code was originally in: http://gallery.pangeo.io/repos/xgcm/xgcm-examples/01_eccov4.html
I ahve been trying to migrate this into Pythia cookbooks here: https://github.com/ProjectPythiaCookbooks/xgcm-cookbook/pull/2
Drew Camron (Jun 29 2022 at 16:15):https://github.com/pangeo-data/pangeo/issues/774 (it's a "requester pays" bucket and you have to set up auth for that, even on free credits)
Max Grover (Jun 29 2022 at 16:47):Yeah - Ryan mentioned that they moved to that after getting $1000s of charges they moved towards that model..
John Clyne (Jun 29 2022 at 17:17):@Brian Rose , @Robert Ford , there may be a solution to the ocean data access issue. @Max Grover has some additional information, and some discussion is required. How about we gather in ML-475 at 1330 MT to discuss?
The Meets link for anyone that wants to join remotely is meet.google.com/uhn-gepw-gcv
Kevin Paul (Jun 29 2022 at 17:27):All: Because there appear to be issues creating forks from the ProjectPythiaCookbooks/projectpythiacookbooks.github.io repo when that repo is a fork of the ProjectPythiaTutorials/projectpythiatutorials.github.io, I've asked GitHub to detach the fork from the original repo. Just FYI. When the process is complete, any existing forks of ProjectPythiaCookbooks/projectpythiacookbooks.github.io may be messed up.
I am hoping not, but I don't know.
Kevin Paul (Jun 29 2022 at 17:29):Well, I was just told the repos have been detached and existing forks of the Cookbooks repo have been moved appropriately. Yay!
Orhan Eroglu (Jun 29 2022 at 19:01):Drew Camron said:
https://github.com/pangeo-data/pangeo/issues/774 (it's a "requester pays" bucket and you have to set up auth for that, even on free credits)
Thanks for this!
Brian Rose (Jun 29 2022 at 19:21):To all who are developing Cookbook repos: I've just opened PRs on all the existing Cookbook repos that updates the build infrastructure. These farm out some of the GitHub Actions workflows to a centralized repo which will be easier to maintain.
I tagged individuals for review who seem to be the folks "leading" each repo. You should be good to merge those changes as long as the tests are passing, but let me know if you see anything funny!
Robert Ford (Jun 29 2022 at 20:04):https://github.com/ProjectPythiaCookbooks/.github/issues/2 @Julia Kent
Brian Rose (Jun 29 2022 at 21:57):All, we'll try to take a group photo / screenshot during our wrap-up meeting this afternoon! Be there!
Julia Kent (Jun 29 2022 at 22:02):We're syncing up now
John Clyne (Jun 29 2022 at 22:45):Hi All,
We are wrapping up the Hackathon! A couple of quick things before folks drift back to their day jobs :-)
Thanks for all of your efforts over the last three days! We've generated 7 cookbooks and a whole bunch of infrastructure! It was a super successful event.
One small ask: if you've transferred a cookbook to the ProjectPythiaCookbooks org could you please open an issue on your repo that summarizes the state, particularly with regard to todo items to finalize the cookbook?
Thanks!
Julia Kent (Jul 07 2022 at 15:44):I just got an email asking for an example of concatenating datafiles. Do we have this covered in a Cookbook somewhere? Would love to send a Project Pythia example.
Michaela Sizemore (Jul 07 2022 at 16:22):I'm not sure about a Cookbook/Project Pythia example, or if this will really cover what the user is looking for, but I had to combine two wrfout files into one for a GeoCAT-example due GitHub file size limits. The steps were creating an array with necessary variable names, reading in the two files with xarray, selecting the data variables, then creating and writing to a new/third datafile and using that file in the example. Again, not sure if it is fully what they would be looking for, but if it's something, you can find the example on the GeoCAT-examples repo here: https://geocat-examples.readthedocs.io/en/latest/gallery/WRF/NCL_wrf_interp_3.html#sphx-glr-gallery-wrf-ncl-wrf-interp-3-py
Julia Kent (Jul 07 2022 at 16:23):Thanks @Michaela Sizemore! I linked them to some Xarray documentation on combining data and that seemed sufficient for now
Julia Kent (Jul 07 2022 at 16:24):But I do think we want some more examples of this somewhere for different types of data in the Pythia world
Last updated: May 16 2025 at 17:14 UTC