Helm & Argo CD

Note

Helm is required to follow along with this documentation. If you don’t have Helm installed please follow the documentation here to install it.

Helm is a package manager for k8s. It refers to k8s packages as charts. Charts are a bundle of YAML definitions required to create an instance of a k8s application. Values can be set as variables in the YAML files which allow for more customization of charts and easier sharing and reproducibility. A chart has been created to deploy web based visualization applications and it can be used to define custom deployments based on user input.

Note

For web based applications hosted on the CISL cloud there is a Helm chart example here you can clone and utilize to create a chart for your application quickly. The included Jupyter Notebook file contains code that will create a custom values.yaml file based on user input. It will also run the helm install command for you as well. You do need a kubeconfig file and a namespace to run the example.

Creating Helm Charts

The Helm Getting Started guide is an excellent resource when it comes to understanding how to create new Helm charts. The helm command includes an easy way to create a new Helm chart template with the following command:

helm create myproject

This command creates a new folder called myproject that contains a few files to start. For most use cases all the files are going to be removed or replaced with the content needed to deploy the objects that we’re interested in. To start let’s remove all the files created in the templates directory.

rm -rf myproject/templates/*

Instead of using the default files, the files required to deploy the k8s object for the application being deployed, a web app in this case, are going to be created and used instead. The web app being deployed in this case requires 3 different templates in total, deployment.yaml, service.yaml, and ingress.yaml.

values.yaml file

The values.yaml file contains specific values to use in the Helm chart templates that are created. Typically the files created in the myproject/templates/ directory will use variables defined in double curly braces, {{ }}, to reference the variables set in the values.yaml file. These values can also be overridden in the CLI command with the --set flag. The syntax used to define and utilize the variables and values.yaml file will be covered in more detail in the next sections.

A list of values used in a Helm chart can be viewed by running the following command:

helm show values myproject

deployment.yaml file

A Deployment is where the state of Pods are provided and maintained to the specifications declared in the object definition.

The deployment.yaml file is where the requirements for the application object are defined. This is where the application is named, the image to use is defined, the port to expose is selected, and any other application customizations are set. An example of a deployment.yaml file is as follows:

apiVersion

Define the Kubernetes API version to use for the object kind: declared.

kind:

Define the type of kubernetes object that is going to be declared.

metadata:

Define the object metadata such as the Deployment name, labels to apply, and the k8s namespace to deploy to.

spec:

Define the resources for the k8s object.

replicas:

Define the number of containers to run.

selector:

Identify the set of objects to use in the deployment based on labels set in metadata.

template:

Define the Pod specification. In a Deployment this is a nested version of the Pod template.

labels:

Define the labels to apply to the pods for use with the label selector.

containers:

Define what container resources to use in the Pod.

image:

Define the container image to deploy. By default this looks to Docker Hub

resources:

Define the container resource requests and limits.

ports:

Define the ports that are exposed in the container and what ports to expose to k8s.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.webapp.name }}
  namespace: {{ .Release.Namespace }}
  labels:
    app: {{ .Values.webapp.name }}
    group: {{ .Values.webapp.group }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ .Values.webapp.name }}
  template:
    metadata:
      labels:
        app: {{ .Values.webapp.name }}
    spec:
      containers:
      - name: {{ .Values.webapp.name }}
        image: {{ .Values.webapp.container.image }}
        resources:
          limits:
            memory: {{ .Values.webapp.container.memory }}
            cpu: {{ .Values.webapp.container.cpu }}
        ports:
        - containerPort: {{ .Values.webapp.container.port }}

As mentioned previously this file utilizes variables defined in the values.yaml file and ones specified inline when running the helm install command. Looking at the first variable defined in name: one can breakdown how these variables are defined in the values.yaml file. To start the full variable is {{ .Values.webapp.name }}. The .Values start tells Helm to look in the values.yaml file. It then specifies the webapp line of that file and then under webapp to select the value assigned to name. A full example of a values.yaml file can be seen below:

replicaCount: 1

webapp:
  name: ncote-helm-test
  group: ncote-helm-test
  path: /viztest
  tls:
    fqdn: ncote-helm-test.k8s.ucar.edu
    secretName: incommon-cert-ncote-helm-test
  container: 
    image: ncote/ncar-viz
    port: 5001
    memory: 1G
    cpu: 2

The variables are defined by referencing the different indented layers of the file. For example if we wanted to supply our template files with the container imaged used in the deployment object we would specify it with the following {{ .Values.webapp.container.image }}. For base values like replicaCount in the example we would only have to supply {{ .Values.replicaCount }}. There is an outlier in the {{ .Release.Namespace }} variable because the namespace to utilize for the deployment must be specified in the CLI command and is then fed to this placeholder.

The full list of specific variables defined in the above values.yaml file is as follows:

• .Values.replicaCount

• .Values.webapp.name

• .Values.webapp.group

• .Values.webapp.path

• .Values.webapp.tls.fqdn

• .Values.webapp.tls.secretName

• .Values.webapp.container.image

• .Values.webapp.container.port

• .Values.webapp.container.memory

• .Values.webapp.container.cpu

Note

These values can be supplied inline with the following flag helm install --set webapp.name=example --set webapp.path=/example. It is recommended to create a values.yaml specific to your application for this instead of supplying the values inline.

service.yaml file

A Service in k8s is a method to expose a network application that is running in one or more Pods.

An example of a service.yaml file is as follows:

apiVersion: v1
kind: Service
metadata:
  name: {{ .Values.webapp.name }}
  namespace: {{ .Release.Namespace }}
  labels:
    group: {{ .Values.webapp.group }}
spec:
  ports:
  - port: {{ .Values.webapp.container.port }}
  selector:
    app: {{ .Values.webapp.name }}

This reuses the values supplied in the Deployment file. This is to map the Deployment/Pods directly to the Service and provide the Service with the network specifications.

ingress.yaml file

A k8s Ingress object is one that maps an application exposed by a Service and routes access to external resources outside the k8s cluster, typically this uses HTTP(s)

annotations:

Annotations are used to attach non-identifying metadata to the object. This means that annotations are not used to identify and select objects. In this example annotations are used to declare a configuration option to apply to the object.

cert-manager.io:

cert-manager.io/cluster-issuer is used to provide the Ingress object information on how to request and obtain certificates as configured in the cluster. The cluster we use has a certificate issuer named incommon that is used to provide certificates for web applications in the *.k8s.ucar.edu domain.

ingressClassName:

This is where the name of the ingress controller is provided. The cluster we use has a ingress controller named nginx and that is used to control the different Ingress objects deployed.

tls:

The tls: field is where we define the hostname, in hosts:, to use in the TLS certificate provided as well as set a secretName for the certificate issued.

Note

The secretName field needs to be unique for the fully qualified domain name (FQDN) used since it is tied to the TLS certificate that is tied to the same FQDN.

host:

This is the FQDN for the application being deployed. This name needs to be unique and also has to be in the .k8s.ucar.edu subdomain. The cluster utilizes ExternalDNS to provision new DNS records but it is limited to the .k8s.ucar.edu domain.

http:

Define how to expose the service to HTTP(S) traffic.

paths:

Paths are where endpoints are defined in order to access the applications. A hostname can have multiple paths defined by multiple ingresses running multiple applications.

backend:

Define the Service to connect the Ingress to by providing the Service name and the port exposed by the Service. This should match exactly what was included in the Service manifest.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ .Values.webapp.name }}
  namespace: {{ .Release.Namespace }}
  labels:
    group: {{ .Values.webapp.group }}
  annotations:
    cert-manager.io/cluster-issuer: "incommon"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
      - {{ .Values.webapp.tls.fqdn }}
      secretName: {{ .Values.webapp.tls.secretName }}
  rules:
  - host: {{ .Values.webapp.tls.fqdn }}
    http:
      paths:
      - path: {{ .Values.webapp.path }}
        pathType: Prefix
        backend:
          service:
            name: {{ .Values.webapp.name }}
            port:
              number: {{ .Values.webapp.container.port }}

Note

The values for cert-manager.io/cluster-issuer and ingressClassName are not variables and instead are explicitly defined. These should not be changed if you are using the NCAR | CISL provided k8s cluster. If you are deploying these to a different cluster you will want to match the cert-manager and ingress controller name to the cluster you are deploying to.

There’s a few important values to call out in the ingress.yaml file that have not been talked about so far. The first new variable is {{ .Values.webapp.tls.fqdn }} and this is the fully qualified domain name (FQDN) for the application. The NCAR provided k8s cluster utilizes ExternalDNS to assign a DNS record to the Ingress and ultimately the application being exposed.

Note

The ExternalDNS provided can only create new records in the .k8s.ucar.edu domain. It can not provision new subdomains. The FQDN value should be set to a hostname that is descriptive of your service followed by .k8s.ucar.edu.

The next important value is also in the tls field and it is {{ .Values.webapp.tls.secretName }}. This secret is the location of the TLS certificate assigned to the FQDN requested by the ingress. This value should be unique to a FQDN, but can be shared among different ingress objects that use different paths on top of the FQDN. Speaking of paths the next value to discuss is {{ .Values.webapp.path }}. This is what is appended at the end of your FQDN. For example a path value of / on top of example.k8s.ucar.edu would give the ingress a URL of [https://example.k8s.ucar.edu/] while a path value of /test would give the ingress a URL of https://example.k8s.ucar.edu/test.

Note

The NCAR | CISL k8s clusters ingress controller is setup to only provide HTTPS traffic over port 443 and utilizes cert-manager to apply valid TLS certificates to supply a trusted and secure connection to the applications exposed.