Getting started#

This Documentation site was built with Jupyter Book and utilizes the Sphinx Pythia Theme.

Clone the GitHub repository#

In order to do development on the site the GitHub repository should be cloned to the users local machine.

Note

All changes and improvements should be coded and tested locally before being pushed back up to the repository. Any changes to docs/ that are pushed will be automatically updated.

git clone https://github.com/NCAR/cisl-cloud.git
# Change directory to the Jupyter Book base
cd docs

CI/CD Process#

Any changes pushed to the docs/ folder in the main branch of the cisl-cloud repository will kick off a full CI/CD workflow to update your changes automatically. This link to docs-cicd.yaml is the definition for the workflow that is kicked off. It runs on Ubuntu whenever there is a push to the docs/ path and has two total jobs. The workflow steps that occurs on a push are outlined below.

Build HTML, update GitHub pages, Build Docker image#

The first job is to build all the documentation. This includes creating all the HTML files with jupyter-book build, pushing the updates to GitHub Pages with ghp-import, and finally creating a Docker image that includes all the HTML files created with jupyter-book build. The job that accomplishes all of this is called build-docs and consists of 9 steps that are outlined below.

  1. Checkout the repo on the Ubuntu machine

  2. Setup Python

  3. Install Python dependencies

  4. Run jupyter-book build

  5. Run ghp-import to update GitHub pages

  6. Login to Docker Hub

  7. Extract metadata for Docker

  8. Get the current date

  9. Build and push the Docker image

Running ghp-import will push changes to the gh-pages branch of our GitHub repository. When this push occurs it kicks off the default job to build the website hosted on GitHub pages. We utilize the current date for the Docker Image tag along with latest. We use the date tags in production and in our Helm charts. Once the new Docker image is built we need to update the Helm chart used to deploy the application to K8s so that it is utilizing the new image.

Update Helm#

Updating the Helm chart with the new image tag is the second job in the workflow and consists of 6 steps.

  1. Checkout the main branch of the repo on the Ubuntu machine

  2. Get the current date

  3. Run sed to update the values.yaml file

  4. Run sed to update the Chart.yaml file

  5. Run a python script to update the version in Chart.yaml by 1

  6. Commit and push changes

The sed commands remove the lines at the end of each file that include the image to use and replace it with the new image tag. The python script was created to update the version number in the middle of the Chart.yaml file by one to match semantic versioning scheme.

Argo CD to K8s#

Once the Helm chart is updated in GitHub then our instance of Argo CD will become out of sync with the repository. When a project is set to update automatically in Argo CD it runs a sync against the configured repository every 3 minutes by default. When the status becomes out of sync for the cloud documentation it is set to automatically update. This will use the new Helm chart, that includes the new image tag, and roll out those changes to K8s.

The documentation now has been updated on K8s and GitHub pages for internal and external viewing. Both sites will also be in sync with this process.

Manual Process#

Build the HTML files#

Jupyter Book documentation is laid out with iPython Notebooks and YAML files. There are other allowed types, link to allowed file types, but for this project those are the two main file types in use. Jupyter Book provides a method to create all the associated HTML content with the following CLI command:

jupyter-book build .

If there are any errors during the build process they will be part of the output from the build command. Please address any errors before continuing development.

Note

If there is code in the notebook being added that should not be executed during build it needs to be added to the exclude_patterns patterns sections of the _config.yml file.

The build process creates a new directory named ‘_build’ in the current workspace. In the _build directory is an HTML directory that contains all the content needed to host this as a static webpage.

Testing locally#

Once the HTML files are built and placed in the _build directory the site can be launched and explored for testing purposes prior to merging changes in to the code repository. This can be accomplished by directing your browser to the files locally. In Linux this can be accomplished via the command line, as long as firefox is installed, with the following:

firefox _build/html/index.html

This can also be accomplished by pasting the full path to the index.html file directly into the browser window. For example the following works via Chrome on a Windows system when the files are hosted on Windows Subsystem for Linux: file://wsl.localhost/Ubuntu/home/ncote/Code/CCPPdocs/jupyter-docs/_build/html/index.html

Push to GitHub#

When the local testing is complete and results are as expected it’s time to push any changes to the GitHub repository before updating GitHub pages. This is important because any images referenced won’t be in the repository until the updates are pushed to the GitHub repo.

Note

There is a GitHub action setup to build a Docker image whenever there is a push to the docs/_build/html/ directory. This image will be pushed to the cislcloudpilot Docker Hub with a tag of :latest by default. More information on how the Docker image is hosted on k8s can be found here.

GitHub Pages#

There’s a few options on how to deploy to GitHub pages on the JupyterBook documentation. This is going to cover using ghp-import as it eliminates some manual steps.

In this scenario the documentation repository here has already been cloned and the command line is in the jupyter-docs/ directory.

  1. Install ghp-import

pip install ghp-import

Note

ghp-import is also included in the requirements.txt file for the repository.

  1. Build book updates Make sure any changes that have been made to the content have been built. As always, any changes should be tested locally before being pushed to GitHub.

jupyter-book build .

Note

Changes to custom css aren’t always picked up. If this happens remove the _build/ directory locally. It will be updated fully when the next jupyter-book build . command is run.

  1. Run ghp-import From the command line, in the documentation root directory (contains the _build/html/ directory), run the following command:

ghp-import -n -p -f _build/html

At this point GitHub will kick off a new action to rebuild and deploy the page. The status of that job can be viewed here. Once the run has completed successfully changes should be reflected on the live site.

On this page