Appliance-Based Install Notes

The following are notes on deploying OCP using the Appliance Builder and Agent Based Installer.

Warning

Currently the following process works with 4.14.x and beyond.

Build the disk image

Set the environment variables.

Important

Use absolute directory paths.

Tip

For container info and download instructions see: OpenShift-based Appliance Builder

cat << EOF > ./variables

export APPLIANCE_IMAGE="quay.io/edge-infrastructure/openshift-appliance:latest"
export APPLIANCE_ASSETS="/home/vince/OCP/appliance-builder"
EOF

source ./variables

Get the openshift appliance builder.
```
podman pull $APPLIANCE_IMAGE
```
Generate the appliance template. A new file, “appliance-config.yaml” is created in the $APPLIANCE_ASSETS directory.
```
podman run --rm -it --pull newer -v $APPLIANCE_ASSETS:/assets:Z $APPLIANCE_IMAGE generate-config
```

Modify “appliance-config.yaml” for your environment.

Important

diskSizeGB is somewhat confusing. I’m unable to build this without the option so not optional. I also discovered, depending on the mirror sizing, finding the right size was hit or miss. With the following ApplianceConfig I required 300Gib.

apiVersion: v1beta1
kind: ApplianceConfig
ocpRelease:
  # OCP release version in major.minor or major.minor.patch format
  # (in case of major.minor - latest patch version will be used)
  # If the specified version is not yet available, the latest supported version will be used.
  version: 4.16
  # OCP release update channel: stable|fast|eus|candidate
  # Default: stable
  # [Optional]
  channel: stable
  # OCP release CPU architecture: x86_64|aarch64|ppc64le
  # Default: x86_64
  # [Optional]
  cpuArchitecture: x86_64
# Virtual size of the appliance disk image.
# If specified, should be at least 150GiB.
# If not specified, the disk image should be resized when
# cloning to a device (e.g. using virt-resize tool).
# [Optional]
diskSizeGB: 300
pullSecret: <your-pull-secret>
sshKey: <your-ssh-key>
userCorePass: <your-core-passwd>
imageRegistry:
  # Default: docker.io/library/registry:2
  # [Optional]
  uri: docker.io/library/registry:2
  # Default: 5005
  # [Optional]
  port: 5005
# Enable all default CatalogSources (on openshift-marketplace namespace).
# Should be disabled for disconnected environments.
# Default: false
# [Optional]
enableDefaultSources: false
# Stop the local registry post cluster installation.
# Note that additional images and operators won't be available when stopped.
# Default: false
# [Optional]
stopLocalRegistry: false
# Additional images to be included in the appliance disk image.
# [Optional]
additionalImages:
  - name: registry.redhat.io/ubi8/ubi:latest
  - name: registry.redhat.io/ubi9/ubi:latest
  - name: registry.redhat.io/ubi9/httpd-24:latest
  - name: registry.redhat.io/ubi9/nginx-122:latest
  - name: registry.redhat.io/rhel8/support-tools:latest
  - name: registry.redhat.io/rhel9/support-tools:latest
  - name: registry.redhat.io/openshift4/dpdk-base-rhel8:latest
  - name: registry.redhat.io/openshift4/ose-cluster-node-tuning-rhel9-operator:v4.16
  - name: ghcr.io/k8snetworkplumbingwg/sriov-network-device-plugin:latest
  - name: quay.io/openshift-scale/etcd-perf:latest
  - name: docker.io/centos/tools:latest
  - name: docker.io/f5devcentral/f5-hello-world:latest
  - name: docker.io/library/httpd:latest
  - name: docker.io/library/nginx:latest
# Operators to be included in the appliance disk image.
# See examples in https://github.com/openshift/oc-mirror/blob/main/docs/imageset-config-ref.yaml.
# [Optional]
operators:
- catalog: registry.redhat.io/redhat/redhat-operator-index:v4.16
  packages:
    - name: advanced-cluster-management
    - name: cincinnati-operator
    - name: kubernetes-nmstate-operator
    - name: kubevirt-hyperconverged
    - name: local-storage-operator
    - name: lvms-operator
    - name: metallb-operator
    - name: multicluster-engine
    - name: odf-operator
    - name: openshift-gitops-operator
    - name: quay-operator
    - name: skupper-operator
    - name: sriov-network-operator
    - name: topology-aware-lifecycle-manager

Build the disk image. This will create a “raw” disk image for your cluster appliance.

Important

To successfully run these commands you have to sudo.
```
sudo podman run --rm -it --pull newer --privileged --net=host -v $APPLIANCE_ASSETS:/assets:Z $APPLIANCE_IMAGE build
```
Tip

If needed you can rebuild the disk image with another version or updated or additional manifests but you must first “clean” the assets directory first. The clean command keeps the cache folder under assets intact. To clean the entire cache as well, add the --cache flag with the clean command.
```
sudo podman run --rm -it -v $APPLIANCE_ASSETS:/assets:Z $APPLIANCE_IMAGE clean
```

Clone the appliance disk image

In my environment I’m using libvirt.

Convert the raw image to qcow2.

qemu-img convert -O qcow2 appliance.raw appliance-4.14.30.qcow2

Create a disk image for each node and copy to the destination storage pool. In my case 3 nodes host11-13.
```
for i in {11..13}; do sudo cp appliance-4.14.30.qcow2 /local/host$i.qcow2; done;
```

Tip

For baremetal you can copy the raw image to the destination drive

dd if=appliance.raw of=/dev/sda bs=1M status=progress

Create the agent install manifests

To create the install manifests follow the instructions found here: Agent-Based Install Notes

With “openshift-install” run the following command. In my case I’m using a “workdir” dir to supply the required yaml files.

Important

The big difference between this method and Agent-Based is the command syntax. Substituting the command switch “image” for “config-image”.
```
./openshift-install agent create config-image --dir ./workdir
```
Note

This is not a bootable image. It contains all the necessary information to build the cluster. The boot image is contained on the disk image created earlier.

Copy config-image iso to VM’s

for i in {11..13}; do sudo cp ./workdir/agentconfig.noarch.iso /local/host$i-agentconfig.iso; done;

Boot the VM’s with the ISO created in the previous step. Follow the progress with the following command:
```
./openshift-install agent wait-for install-complete --dir ./workdir
```