Skip to content

Edge Lab

Please see http://edgelab.digital for complete the reference architecture and additional documentation.

Assumptions:

  • the leader node have internet access for downloading containerd and k3s binaries
  • all nodes have have internet access for downloading containers
  • drpcli has been installed on client system

Requirements:

  • Edge Lab requires Digital Rebar Platform tip (post v4.2) due to RPi install requirements.
  • K3s has been migrated to use the standardized kube-lib content pack instead of a specialized k3s installation. This also uses the v4.6 cluster pattern where the DRP server acts as the cluster manager.

Building & Installing DRP on Edge Lab

Please see the documentation in https://gitlab.com/rackn/edgelab for building the physical environment.

Setup Edge Lab Content

Note: make sure that you export RS_ENDPOINT to your endpoint!

Edge Lab (edge-lab) is available in the RackN Catalog, you can install it using drpcli catalog item install edge-lab This was done for you as part of the bootstrapping process.

To manually create and upload:

  1. clone edge-lab with DRP content: git clone https://gitlab.com/rackn/provision-content
  2. got to the edge-lab content: cd provision-content\edge-lab
  3. build the content pack: drpcli contents bundle edge-lab.yaml
  4. install the edge-lab content pack: drpcli contents upload edge-lab.yaml

If you make changes, you can repeat steps 3 and 4 to upload the latest content.

Setup K3s Cluster via Kube-Lib

This process uses the standard kube-lib module. Use catalog item install kube-lib if it is missing.

Recommended, install kubectl has been installed on your client system.

Note

You must have attached storage in each node for the k3s cluster to operate! This could be an SDcard or USB drive. If the partitions are not xfs, the code will wipe the drive!

By default, the cluster/filter will select ALL machines in the edge-lab cluster. To build your cluster:

  1. select the edge-lab-manager machine (which is also the DRP server)
  2. apply the k3-cluster-build workflow

  3. wait for workflow to complete on cluster manager

    1. this will select and start the k3s install on the leader
    2. after the leader is installed, this will install k3s on all the remaining machines as workders

  4. use drpcli profiles getedge-lab-managerparam kube-lib/kube-config --decode > kube.conf to get the kube.config file. You can also download from the cluster's profile in the UX.

To access on the DRP server or cluster:

  1. test with kubectl --kubeconfig=kube.conf get nodes

To access from another system, use an SSH tunnel:

  1. setup an ssh tunnel: ssh root@[host wifi ip] -L 6443:[leader node ip]:6443
  2. modify your admin.conf file to use 127.0.0.1 instead of the leader node ip
  3. test with kubectl --kubeconfig=kube.conf get nodes

Access Kubernetes Dashboard

The Kubernetes Dashboard v2 is automatically installed by the k3s-cluster-manager workflow. Admin and service accounts are created for access.

To access the Dashboard:

  1. use kubectl --kubeconfig=kube.conf proxy to create a network path to your cluster.
  2. get the Dashboard authentication token drpcli profiles get edge-lab-manager param kube-lib/dashboard-token --decode
  3. open the Dashboard using the proxy: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

Install OpenFaaS via Helm

The k3s install above will automatically include Helm v3 and run charts defined in the kube-lib/helm-charts param array.

OpenFaaS is provided as an example in the helm-openfaas profile. To include OpenFaaS in the k3s install, simply assign the helm-openfaas profile on cluster leader based on your architecture (you can assign it to all cluster machines and it will be ignore on the non-leaders).

Since the helm stages are idempotent, either include the profile before running the k3s-install workflow or re-run the workflow.

The OpenFaaS helm chart install will write its password to the cluster profile using openfaas/password. To retrieve the password use drpcli profiles get cluster01 param openfaas/password --decode.

Machine Reset

Setting a machine's workflow to edge-lab-reboot-to-discover will force the machine through a soft reboot process and back to discovery.

To indicate that the machine has been rebooted, the icon is set to the red recycling image. The icon will remain until it get's reset.

The machine is avilable once it is marked Runnable again.

Edge Lab Bootstrap

This content pack includes the steps needed to bootstrap the Edge Lab server. This includes a workflow, stage, and tasks to:

  1. Download the required isos (sledgehammer rpi, but others as well).
  2. Set the default preferences to make Edge Lab work.
  3. Set up an ssh key pair if one does not exist and update the global profile with it. This can be used to access the clients from the server.
  4. Set up the storage attach Param for the Edge Lab
  5. Create the default subnet for the Edge Lab.

The start_me.sh script will install the edge-lab content pack and set the DRP server's workflow to the bootstrap workflow. The status of bootstrapping can be tracked in the jobs for the DRP server.