Installing OpenShift using Azure Marketplace

Red Hat OpenShift Container Platform is now available through the Microsoft Azure Marketplace. This allows you to create self-managed OpenShift Container Platform cluster deployments which are billed on pay-per-use basis (hourly, per core) with your Azure subscription while still being supported by Red Hat directly. This is done by using a private RHCOS image that is billed on an hourly basis through your Azure subscription. Control plane nodes and infrastructure nodes run public RHCOS images and do not incur OpenShift subscription costs. This document explains how to install such a cluster and take advantage of this billing model.

Understanding the installation process

This guide assumes basic familiarity with the overall design concepts and installation procedure of Red Hat OpenShift Container Platform. For a general introduction, please refer to our product documentation.

The deployment of OpenShift Container Platform via the Azure Marketplace is conducted using the same utilities and procedures as a regular self-managed OpenShift Container Platform installation. The setup process using the Azure Marketplace image is identical to a regular Azure installation; both installer-provisioned infrastructure and user-provisioned infrastructure scenarios are supported.

If you have never deployed OpenShift Container Platform on Azure before, you can explore the Install OpenShift on Azure quickly chapter in the installation guide. Customized installations are also supported; in fact, this guide requires that you customize installation parameters to take advantage of the billing integration/benefits.

Limitations

The following scenarios are not supported for installing OpenShift Container Platform using the Azure Marketplace:

Single-node deployments
Three-node deployments

Setting up an OpenShift Container Platform cluster using Azure Marketplace

For the cluster to use the Azure marketplace images, you must have the following utilities available:

Azure CLI client (az)
OpenShift installation program (openshift-install)

If you are not already authenticated with your Azure CLI client, log in to the account that you requested to be entitled for use of OpenShift Azure Marketplace images.

Selecting the appropriate image

Use the Azure CLI to find the appropriate marketplace image. There are three different OpenShift offers available:
- OCP: rh-ocp-worker
- OPP: rh-opp-worker
- OKE: rh-oke-worker
  
  In the following commands, use one of these offer choices consistently:
To display all available OpenShift images:
```
# For OCP
$  az vm image list --all --offer rh-ocp-worker --publisher redhat

# For OPP
$  az vm image list --all --offer rh-opp-worker --publisher redhat

# For OKE
$  az vm image list --all --offer rh-oke-worker --publisher redhat
```
Inspect the image for your offer. Note that this 4.8 RHCOS image must be used even for 4.9 installations. In the case of 4.9, the image will be automatically upgraded.

Note: Users in the EMEA region should use the publisher redhat-limited instead of redhat in all instances throughout this document.
```
# For OCP
$ az vm image show --urn redhat:rh-ocp-worker:rh-ocp-worker-a:4.8.2021122100

# For OPP
$ az vm image show --urn redhat:rh-opp-worker:rh-opp-worker-a:4.8.2021122100

# For OKE
$ az vm image show --urn redhat:rh-oke-worker:rh-oke-worker-a:4.8.2021122100
```
Note: The SKUs used here are for Generation 2 VM images. The default instance types used in OpenShift are Gen2-compatible.

If you choose to use an instance type which is only Gen1-compatible, you can use a Gen1 image by appending -gen1 to your SKU. For example, the Gen1 equivalent for the Gen2 rh-ocp-worker SKU is rh-ocp-worker-gen1. If you choose to use an instance type which is only Gen2-compatible in a pre-4.11 cluster, the instance type must be specified in the .spec.template.spec.providerSpec.value.vmSize field when editing the machineset manifests in the following section. A Gen2-only instance type can only be specified in the install config for 4.11+ clusters.

Review and accept the usage terms of the image using the Azure CLI.

Review the terms for the image offering:

# For OCP
$ az vm image terms show --urn redhat:rh-ocp-worker:rh-ocp-worker:4.8.2021122100

# For OPP
$ az vm image terms show --urn redhat:rh-opp-worker:rh-opp-worker:4.8.2021122100

# For OKE
$ az vm image terms show --urn redhat:rh-oke-worker:rh-oke-worker:4.8.2021122100

Accept the terms provided by this image offering:

#For OCP
$ az vm image terms accept --urn redhat:rh-ocp-worker:rh-ocp-worker:4.8.2021122100

#For OPP
$ az vm image terms accept --urn redhat:rh-opp-worker:rh-opp-worker:4.8.2021122100

#For OKE
$ az vm image terms accept --urn redhat:rh-oke-worker:rh-oke-worker:4.8.2021122100

Specifying Marketplace Images in the Install Config (only available in 4.11)

4.11 clusters can specify Azure Marketplace images in the install config. It is not necessary to edit the machineset manifests, and the next section may be skipped if you specify images in the install config. To specify marketplace images in the install config, edit installconfig.compute.platform.azure.osImage:

    # For OCP
    compute:
    - name: worker
      platform:
        azure:
          osImage:
            publisher: redhat
            offer: rh-ocp-worker
            sku: rh-ocp-worker
            version: 4.8.2021122100

    # For OPP
    compute:
    - name: worker
      platform:
        azure:
          osImage:
            publisher: redhat
            offer: rh-opp-worker
            sku: rh-opp-worker
            version: 4.8.2021122100

    # For OKE
    compute:
    - name: worker
      platform:
        azure:
          osImage:
            publisher: redhat
            offer: rh-oke-worker
            sku: rh-oke-worker
            version: 4.8.2021122100

Modifying the installation manifests to use Azure Marketplace images (4.10 & earlier)

These steps demonstrate how to use openshift-install to provision a cluster with compute nodes using marketplace images. If you plan on provisioning nodes using ARM templates, you can skip ahead to the section Using ARM Templates to Provision Nodes with Azure Marketplace Images.

Generate the installation manifests:

$ openshift-install create manifests --dir <installation_dir>

Edit the .spec.template.spec.providerSpec.value.image property of the compute machine set definitions with the appropriate values offer, publisher, sku, and version from the previous section. These are the three files that must be updated:
- <installation_dir>/openshift/99_openshift-cluster-api_worker-machineset-0.yaml
- <installation_dir>/openshift/99_openshift-cluster-api_worker-machineset-1.yaml
- <installation_dir>/openshift/99_openshift-cluster-api_worker-machineset-2.yaml
In each file, unset the .spec.template.spec.providerSpec.value.image.resourceID property by deleting the value, and supply an empty value for the property ("").

In each file, set type to MarketplaceWithPlan and set offer, publisher, sku, and version in .spec.template.spec.providerSpec.value.image to the values from the previous section. Using the first machine set as an example; the .spec.template.spec.providerSpec.value.image section should look like this when done:

$ cat <installation_dir>/openshift/99_openshift-cluster-api_worker-machineset-0.yaml | grep -A5 -B1 offer 

    # For OCP
    image:
      offer: rh-ocp-worker
      publisher: redhat
      resourceID: ""
      sku: rh-ocp-worker
      version: 4.8.2021122100
      type: MarketplaceWithPlan

    # For OPP
    image:
      offer: rh-opp-worker
      publisher: redhat
      resourceID: ""
      sku: rh-opp-worker
      version: 4.8.2021122100
      type: MarketplaceWithPlan

    # For OKE
    image:
      offer: rh-oke-worker
      publisher: redhat
      resourceID: ""
      sku: rh-oke-worker
      version: 4.8.2021122100
      type: MarketplaceWithPlan

Completing the cluster creation

There are two ways to complete the cluster installation; what way you choose depends on what functionality or method of installation you are using.

Using installer resource provisioning

Note: To use the installer-provisioned infrastructure installation method, you must use an OCP install version greater than or equal to 4.9.11. Older versions of OpenShift Container Platform do not support this automated integration with Azure Marketplace.

If you are planning on letting the OpenShift Container Platform manage machines with machine sets, and not manage machines manually, you can complete a typical installation with the following command:

$ openshift-install create cluster --dir <installation_dir>

Using ARM templates to provision nodes with Azure Marketplace images

To provision an OpenShift Container Platform cluster using Azure Marketplace images with ARM templates, you can follow the standard documentation and update the 06_workers.json ARM Template.

The virtual machine in 06_workers.json has a storageProfile.imageReference that refers to the image created earlier in the installation process:

Example:

"storageProfile" : {
    "imageReference": {
        "id": "[resourceId('Microsoft.Compute/images', variables('imageName'))]"
  },

You can update the storageProfile.imageReference by deleting the id property and adding the offer, publisher, sku, and version properties for the appropriate image. The modification to the ARM template should look similar to the following examples after the updates are made:

# For OCP
"storageProfile": {
    "imageReference": {
        "offer": "rh-ocp-worker",
        "publisher": "redhat",
        "sku": "rh-ocp-worker",
        "version": "4.8.2021122100"
}

# For OPP
"storageProfile": {
    "imageReference": {
        "offer": "rh-opp-worker",
        "publisher": "redhat",
        "sku": "rh-opp-worker",
        "version": "4.8.2021122100"
}

# For OKE
"storageProfile": {
    "imageReference": {
        "offer": "rh-oke-worker",
        "publisher": "redhat",
        "sku": "rh-oke-worker",
        "version": "4.8.2021122100"
}

You will also need to specify a plan for the virtual machine:

# For OCP
"plan" : {
    "name": "rh-ocp-worker",
    "product": "rh-ocp-worker",
    "publisher": "redhat"
},

# For OPP
"plan" : {
    "name": "rh-opp-worker",
    "product": "rh-opp-worker",
    "publisher": "redhat"
},

# For OKE
"plan" : {
    "name": "rh-oke-worker",
    "product": "rh-oke-worker",
    "publisher": "redhat"
},

Post-cluster creation verification

When the deployment finishes, you have a cluster running with compute nodes using the Azure Marketplace images, and are thus now being billed as they are used through the Azure Marketplace.

Changing machine sets in a running cluster

To change the compute nodes in an existing cluster to use OpenShift Container Platform images from the Azure Marketplace, follow the steps for Modifying a machine set. The values to change in the cluster machine sets are the same as in the Modifying the installation manifests to use Azure Marketplace images step.

Select Your Language

Installing OpenShift using Azure Marketplace

Understanding the installation process

Limitations

Setting up an OpenShift Container Platform cluster using Azure Marketplace

Selecting the appropriate image

Specifying Marketplace Images in the Install Config (only available in 4.11)

Modifying the installation manifests to use Azure Marketplace images (4.10 & earlier)

Completing the cluster creation

Using installer resource provisioning

Using ARM templates to provision nodes with Azure Marketplace images

Post-cluster creation verification

Changing machine sets in a running cluster

Comments

Quick Links

Help

Site Info

Related Sites

About

Red Hat legal and privacy links

Red Hat legal and privacy links

Understanding the installation process

Limitations

Setting up an OpenShift Container Platform cluster using Azure Marketplace

Selecting the appropriate image

Specifying Marketplace Images in the Install Config (only available in 4.11)

Modifying the installation manifests to use Azure Marketplace images (4.10 & earlier)

Completing the cluster creation

Using installer resource provisioning

Using ARM templates to provision nodes with Azure Marketplace images

Post-cluster creation verification

Changing machine sets in a running cluster

Comments

Quick Links

Help

Site Info

Related Sites

Systems Status

About

Red Hat legal and privacy links

Red Hat legal and privacy links