Skip to content

Create a Customer Catalog Repository

This document defines a process on how to properly maintain custom Content Bundles in a Digital Rebar Platform (DRP) managed environment, utilizing a custom Catalog Repository. The Catalog Repository is analogous to an "apt" or "yum" package repository, but stores DRP related Content Bundles. The Catalog Repository becomes an important content source for DRP Multi-Site Manager (MSM) managed infrastructure. DRP Endpoint management is performed with an IaC object type called a VersionSet, which refers to content bundle versions to install, upgrade, or downgrade on a DRP Endpoint. Content not distributed in the RackN master Catalog mirror should be maintained in a custom catalog specific to your organization.

The recommended way to share custom content between the manager and endpoints is to create a custom catalog specific to the manager. This ensures that the main rackn-catalog remains undisturbed and easier to maintain.

After creating a custom catalog, add it to the list of catalog URLs by updating the catalog_urls parameter on the global profile.

Below is a step-by-step guide with basic example content.

Important

Ensure you are on the latest stable release of DRP v4.13.26 or higher for an optimal experience. This version includes many MSM specific improvements and bug fixes.

Step 1: Create Custom Content

In this example, we create a custom content pack with one parameter.

File Structure

test-bundle/
├── params/
│   └── test-bundle-param1.yml
└── meta.yaml
Click to view test-bundle-param1.yml 
---
Name: test-bundle/param1
Description: this is a test param
Documentation: |
This param is a test
Meta:
color: red
icon: box
Schema:
type: string
default: testing
Click to view meta.yaml 
---
Author: RackN
CodeSource: https://gitlab.com/rackn/provision-content
Color: purple
Copyright: RackN 2024
Description: Test Bundle
DisplayName: TestBundle
DocUrl:
Icon: feather alternate
License: APLv2
Name: test-bundle
Order: "500"
Source: RackN
Tags: rackn
Prerequisites: drp-community-content

Step 2: Build custom content bundle

Navigate to the test-bundle directory and run the following command:

drpcli contents bundle test-bundle-v1.1.1.json Version="v1.1.1" --format=json

This will create a test-bundle-v1.1.1.json file which can then be uploaded to DRP.

test-bundle-v1.1.1.json 
{
   "sections": {
      "params": {
         "test-bundle/param1": {
            "Validated": false,
            "Available": false,
            "Errors": [],
            "ReadOnly": false,
            "CreatedBy": "",
            "CreatedAt": "0001-01-01T00:00:00Z",
            "LastModifiedBy": "",
            "LastModifiedAt": "0001-01-01T00:00:00Z",
            "Meta": {
            "color": "red",
            "icon": "box"
            },
            "Endpoint": "",
            "Bundle": "",
            "Description": "this is a test param",
            "Documentation": "this param is a test\n",
            "Name": "test-bundle/param1",
            "Secure": false,
            "Schema": {
               "default": "testing",
               "type": "string"
            }
         }
      }
   },
   "meta": {
      "Author": "RackN",
      "CodeSource": "https://gitlab.com/rackn/provision-content",
      "Color": "purple",
      "Copyright": "RackN 2024",
      "Description": "Testing Bundle",
      "DisplayName": "TestBundle",
      "DocUrl": "",
      "Documentation": "",
      "Icon": "feather alternate",
      "License": "APLv2",
      "Name": "test-bundle",
      "Order": "500",
      "Prerequisites": "drp-community-content",
      "RequiredFeatures": "",
      "Source": "RackN",
      "Tags": "rackn",
      "Type": "dynamic",
      "Version": "v1.1.1"
   }
}

Optionally upload the bundle to DRP

drpcli contents upload test-bundle-v4.20.20.json

Note

The bundle needs to be versioned correctly and created as json. This will make it easier to move this to the right location and then uploaded to a catalog. We use SemVer versioning, with a v prefix in front. SemVer details can be found at https://semver.org/

Step 3: Stage the content

To share your custom bundle effectively, it's recommended to create a custom catalog. This ensures the main rebar-catalog remains organized and unaffected.

The custom catalog folder should be created at the same directory level as the rebar-catalog.

For example: Default catalog contents are located at: /var/lib/dr-provision/tftpboot/files/rebar-catalog The catalog file is located at: /var/lib/dr-provision/tftpboot/files/rebar-catalog/rackn-catalog.json

Note

/var/lib/dr-provision is the default installation location. The operator may have a different location that was specified during installation by the --drp-home-dir option.

Follow These Steps to Stage Your Content

  1. Create a directory for the custom catalog: 
    mkdir -p /var/lib/dr-provision/tftpboot/files/test-catalog
  2. Create a folder named after the custom bundle. !!! Note here that the folder name must match the bundle name. The bundle name is determined by the Name field in the meta.yaml file. In the meta.yaml file for this example, you will see Name: test-bundle. In this example, the bundle is named test-bundle, so the folder will be: 
    mkdir -p /var/lib/dr-provision/tftpboot/files/test-catalog/test-bundle
  3. Move the bundle content file to the appropriate location 
    cp test-bundle-v1.1.1.json /var/lib/dr-provision/tftpboot/files/test-catalog/test-bundle/v1.1.1.json

Note

Ensure the file name matches the version name. The correct path format should be <bundle-name>/<version.json>

Step 4: Create custom catalog

Now that we have the content in the right place, we can go ahead and create a custom catalog for this content.

To ensure that the Meta section of the catalog is created right, you can use a metadata.yaml file. The file needs to be placed in the same location as the catalog (in this case /var/lib/dr-provision/tftpboot/files/test-catalog)

So, for this example, the file will be at /var/lib/dr-provision/tftpboot/files/test-catalog/metadata.yaml

Click here to view metadata.yaml 
Name: test-catalog
Author: Testing
CodeSource: http://localhost:8091/files/test-catalog/test-catalog.json
Color: red
Copyright: Test
Description: Local Test Lab Catalog
DisplayName: Local Test Lab Catalog
DocUrl: http://localhost:8091/files/test-catalog/test-catalog.json
Documentation: |
  Test Catalog.
Icon: book
License: Test.
RequiredFeatures: ""
Source: https://gitlab.com/rackn/example-content
Tags: rackn,catalog,contents,plugins

To create the custom catalog, do the following:

  1. cd /var/lib/dr-provision/tftpboot/files/test-catalog

  2. drpcli catalog build --base-dir=/var/lib/dr-provision/tftpboot/files/test-catalog --catalog-name="test-catalog" --file-url="https://<drp-host>:8090/files/test-catalog"

Note

You need to cd into the directory with the metadata.yaml for it to get picked up.

This will create a file call test-catalog.json in var/lib/dr-provision/tftpboot/files/test-catalog.

Click here to view test-catalog.json 
{
   "meta": {
      "Name": "test-catalog",
      "Version": "v24.10.31-105159",
      "Description": "Local Test Lab Catalog",
      "Source": "https://gitlab.com/rackn/example-content",
      "Documentation": "Test Catalog.\n",
      "RequiredFeatures": "",
      "Prerequisites": "",
      "Color": "red",
      "Icon": "book",
      "Author": "Testing",
      "DisplayName": "Local Test Lab Catalog",
      "License": "Test.",
      "Copyright": "Test",
      "CodeSource": "http://localhost:8091/files/test-catalog/test-catalog.json",
      "Order": "2000",
      "Tags": "rackn,catalog,contents,plugins",
      "DocUrl": "http://localhost:8091/files/test-catalog/test-catalog.json",
      "Type": "",
      "Writable": false,
      "Overwritable": false
   },
   "sections": {
      "catalog_items": {
         "test-bundle-stable": {
            "Validated": false,
            "Available": false,
            "Errors": [],
            "ReadOnly": false,
            "CreatedBy": "",
            "CreatedAt": "0001-01-01T00:00:00Z",
            "LastModifiedBy": "",
            "LastModifiedAt": "0001-01-01T00:00:00Z",
            "Meta": {
               "Author": "RackN",
               "CodeSource": "https://gitlab.com/rackn/provision-content",
               "Color": "purple",
               "Copyright": "RackN 2024",
               "Description": "Test Bundle",
               "DisplayName": "TestBundle",
               "DocUrl": "",
               "Icon": "feather alternate",
               "License": "APLv2",
               "Name": "test-bundle",
               "Order": "500",
               "Prerequisites": "drp-community-content",
               "Tags": "rackn"
            },
            "Endpoint": "",
            "Id": "test-bundle-stable",
            "Type": "catalog_items",
            "Name": "test-bundle",
            "ActualVersion": "v1.1.1",
            "Version": "stable",
            "ContentType": "ContentPackage",
            "Source": "test-bundle/v1.1.1.json",
            "Shasum256": {
               "any/any": "41fcf309212833e4ce35701a4caf5088b8bb3aff7421959935450284d17da98c"
            },
            "NOJQSource": "test-bundle-stable:::test-bundle/v1.1.1.json",
            "Tip": false,
            "HotFix": true
         },
         "test-bundle-tip": {
            "Validated": false,
            "Available": false,
            "Errors": [],
            "ReadOnly": false,
            "CreatedBy": "",
            "CreatedAt": "0001-01-01T00:00:00Z",
            "LastModifiedBy": "",
            "LastModifiedAt": "0001-01-01T00:00:00Z",
            "Meta": {
               "Author": "RackN",
               "CodeSource": "https://gitlab.com/rackn/provision-content",
               "Color": "purple",
               "Copyright": "RackN 2024",
               "Description": "Test Bundle",
               "DisplayName": "TestBundle",
               "DocUrl": "",
               "Icon": "feather alternate",
               "License": "APLv2",
               "Name": "test-bundle",
               "Order": "500",
               "Prerequisites": "drp-community-content",
               "Tags": "rackn"
            },
            "Endpoint": "",
            "Id": "test-bundle-tip",
            "Type": "catalog_items",
            "Name": "test-bundle",
            "ActualVersion": "v1.1.1",
            "Version": "tip",
            "ContentType": "ContentPackage",
            "Source": "test-bundle/v1.1.1.json",
            "Shasum256": {
               "any/any": "41fcf309212833e4ce35701a4caf5088b8bb3aff7421959935450284d17da98c"
            },
            "NOJQSource": "test-bundle-tip:::test-bundle/v1.1.1.json",
            "Tip": false,
            "HotFix": true
         },
         "test-bundle-v1.1.1": {
            "Validated": false,
            "Available": false,
            "Errors": [],
            "ReadOnly": false,
            "CreatedBy": "",
            "CreatedAt": "0001-01-01T00:00:00Z",
            "LastModifiedBy": "",
            "LastModifiedAt": "0001-01-01T00:00:00Z",
            "Meta": {
               "Author": "RackN",
               "CodeSource": "https://gitlab.com/rackn/provision-content",
               "Color": "purple",
               "Copyright": "RackN 2024",
               "Description": "Test Bundle",
               "DisplayName": "TestBundle",
               "DocUrl": "",
               "Icon": "feather alternate",
               "License": "APLv2",
               "Name": "test-bundle",
               "Order": "500",
               "Prerequisites": "drp-community-content",
               "Tags": "rackn"
            },
            "Endpoint": "",
            "Id": "test-bundle-v1.1.1",
            "Type": "catalog_items",
            "Name": "test-bundle",
            "ActualVersion": "v1.1.1",
            "Version": "v1.1.1",
            "ContentType": "ContentPackage",
            "Source": "test-bundle/v1.1.1.json",
            "Shasum256": {
               "any/any": "41fcf309212833e4ce35701a4caf5088b8bb3aff7421959935450284d17da98c"
            },
            "NOJQSource": "test-bundle-v1.1.1:::test-bundle/v1.1.1.json",
            "Tip": false,
            "HotFix": true
         },
         "test-catalog-tip": {
            "Validated": false,
            "Available": false,
            "Errors": [],
            "ReadOnly": false,
            "CreatedBy": "",
            "CreatedAt": "0001-01-01T00:00:00Z",
            "LastModifiedBy": "",
            "LastModifiedAt": "0001-01-01T00:00:00Z",
            "Meta": {
               "Author": "RackN",
               "CodeSource": "https://localhost:8090/files/test-catalog/test-catalog.json",
               "Color": "blue",
               "Copyright": "RackN",
               "DisplayName": "RackN Catalog",
               "DocUrl": "https://localhost:8090/files/test-catalog/test-catalog.json",
               "Icon": "book",
               "License": "RackN",
               "Name": "test-catalog",
               "Order": "1",
               "Prerequisites": "",
               "Tags": "rackn,catalog,contents,plugins"
            },
            "Endpoint": "",
            "Id": "test-catalog-tip",
            "Type": "catalog_items",
            "Name": "test-catalog",
            "ActualVersion": "v24.10.30-133926",
            "Version": "tip",
            "ContentType": "ContentPackage",
            "Source": "https://localhost:8090/files/test-catalog/test-catalog.json",
            "Shasum256": {},
            "NOJQSource": "test-catalog-tip:::https://localhost:8090/files/test-catalog/test-catalog.json",
            "Tip": true,
            "HotFix": false
         }
      }
   }
}

Step 5: Add catalog url to the catalog_urls parameter

To ensure both the default catalog and your custom catalog are available, update the catalog_urls parameter on the global profile (of the manager) with the following command

drpcli profiles set global param catalog_urls to '["{{.ProvisionerURL}}/files/rebar-catalog/rackn-catalog.json", "{{.ProvisionerURL}}/files/test-catalog/test-catalog.json"]'

Note

It is important to note that the `catalog_url` and `catalog_urls` parameters will only be recognized by drpcli 
when there is a successful connection to an endpoint. A standalone drpcli will only pull items from the default 
catalog located at `repo.rackn.io`.

Step 6: Add Content to a Version Set and Apply to an Endpoint

Version sets are used to share content from the manager to various endpoints. It is recommended to maintain a "current state" version set for each endpoint. Follow these steps to create and manage version sets.

Extract the Current State Version Set

  1. Navigate to https://<drp-host>:8092/endpoints.
  2. Select the endpoint you want to update.
  3. Click the Version Set Extract button to create a version set representing the endpoint's current state.
  4. Enter a name for the version set or accept the default, then click Extract.

You can now find the newly extracted version set by navigating to https://<drp-host>:8092/version_sets. If the universal content pack is installed, you’ll also see several pre-defined version sets available.

Create a New Version Set with test-bundle

  1. Navigate to https://<drp-host>:8092/version_sets and click Create.
  2. Enter a valid ID, such as test-bundle-version-set
  3. In the Components field, enter the following JSON to include the test-bundle content: 
    [
 {
   "ActualVersion": "",
   "Name": "test-bundle",
   "Type": "ContentPackage",
   "Version": "v1.1.1"
 }
]
  4. Click Save to create the new version set.

Apply Version Sets to an Endpoint

  1. Navigate back to https://<drp-host>:8092/endpoints and select the endpoint you extracted the version set from. You should now be on the page https://<drp-host>:8092/endpoints/endpoint-name.
  2. Click the plus sign (+) next to Version Sets and select the extracted version set for this endpoint.
  3. Verify the Actions list at the bottom – it should be empty initially.
  4. Click the plus sign (+) again and add the test-bundle-version-set to the endpoint.
  5. Now, the Actions list will display the following action:
  6. AddOrUpdate → ContentPackage → test-bundle
  7. Toggle the Apply button to apply the content to the endpoint.

Note

Version sets can be stacked AND the order matters. Make sure to add the current state first, then introduce the changes. So for this example, you will first add extracted-version-set and then test-bundle-version-set.