Author: Brandon B. Jozsa

This article builds on another post I created this week which provides a basic hypervisor on Red Hat systems, which includes CentOS, Fedora and RHEL. Since this article is a continuation of that series, I would suggest reading through Virtualization Management on RHEL, CentOS, and Fedora - at least as a reference point - before starting to implement this solution.

Part I: Install Requirements

As with other guides, let's start with the installation requirements. Organizing these articles is challenging. I will document client usage in other posts, but for now I am only going to cover the server-side requirements.

First, make sure the following packages are installed:
(Note: use of the -yy flag below)

sudo dnf install bind-utils libguestfs-tools cloud-init -yy
sudo dnf module install virt -yy
sudo dnf install virt-install -yy
sudo systemctl enable libvirtd --now

Part II: Containerized Sushy-Tools

This is where you may find that my instructions differ from other sources you may have encountered, in the fact that I am going to use a container to provide sushy-emulation to the hypervisor host. Considering that I have your attention, proceed with installing Podman:
(Note: use of the -yy flag below)

sudo dnf install podman -yy

Next, you will need to create a configuration file for the sushy-emulator. I'll have you place this into the /etc/ directory to store with any other configs the system is using:

sudo mkdir -p /etc/sushy/
cat << "EOF" | sudo tee /etc/sushy/sushy-emulator.conf
SUSHY_EMULATOR_LISTEN_IP = u'0.0.0.0'
SUSHY_EMULATOR_LISTEN_PORT = 8000
SUSHY_EMULATOR_SSL_CERT = None
SUSHY_EMULATOR_SSL_KEY = None
SUSHY_EMULATOR_OS_CLOUD = None
SUSHY_EMULATOR_LIBVIRT_URI = u'qemu:///system'
SUSHY_EMULATOR_IGNORE_BOOT_DEVICE = True
SUSHY_EMULATOR_BOOT_LOADER_MAP = {
    u'UEFI': {
        u'x86_64': u'/usr/share/OVMF/OVMF_CODE.secboot.fd'
    },
    u'Legacy': {
        u'x86_64': None
    }
}
EOF

Great! Now, run the sushy-emulation container. But first, make sure to verify the container tag being used. To keep things simple, I'll use the latest tag:

export SUSHY_TOOLS_IMAGE=${SUSHY_TOOLS_IMAGE:-"quay.io/metal3-io/sushy-tools"}
sudo podman create --net host --privileged --name sushy-emulator -v "/etc/sushy":/etc/sushy -v "/var/run/libvirt":/var/run/libvirt "${SUSHY_TOOLS_IMAGE}" sushy-emulator -i :: -p 8000 --config /etc/sushy/sushy-emulator.conf

Part III: Remote Sushy-Tools

It's critical to note a couple of important things with the way I've run the container in the previous section. The first is that I am running privileged mode (which isn't really required - in fact it's discouraged), and I am mounting the local /var/run/libvirt directory because I am attaching directly to qemu:///system for local virtual machines. This isn't really a requirement, and there are other options. For example, what if you wanted to manage virtual machines on another hypervisor? This can be done with the following virsh command.

❯ virsh -c 'qemu+ssh://bjozsa@192.168.3.99/system?socket=/var/run/libvirt/libvirt-sock' list --all
 Id   Name         State
-----------------------------
 57   cuttlefish   running
 99   sno002       running
 -    ocp01        shut off
 -    ocp02        shut off
 -    ocp03        shut off
 -    sno004       shut off
 -    tuna         shut off

This means that I can use sushy-tools to connect in this manner as well.

SUSHY_EMULATOR_LISTEN_IP = u'0.0.0.0'
SUSHY_EMULATOR_LISTEN_PORT = 8000
SUSHY_EMULATOR_SSL_CERT = None
SUSHY_EMULATOR_SSL_KEY = None
SUSHY_EMULATOR_OS_CLOUD = None
SUSHY_EMULATOR_LIBVIRT_URI = u'esx://root@galvatron11.jinkit.com?no_verify=1&authfile=/etc/sushy/auth.conf'
# The following is required to be "True" for ZTP installations
SUSHY_EMULATOR_IGNORE_BOOT_DEVICE = True
SUSHY_EMULATOR_BOOT_LOADER_MAP = {
    u'UEFI': {
        u'x86_64': u'/usr/share/OVMF/OVMF_CODE.secboot.fd'
    },
    u'Legacy': {
        u'x86_64': None
    }
}

Part IV: Management for ESXi Guests

I'm not exactly sure if it's lack of need or lack of knowledge, but whatever the reason, I don't see many people talking about Redfish management of ESXi guests. Sushy-Tools actually supports this, and more accurately, this is provided via Libvirt management.

Taking a look at the entry for SUSHY_EMULATOR_LIBVIRT_URI = u'qemu:///system' shows us how this can be accomplished. Using the virsh command, you can manage an ESXi guest today.

ESXI_HOST="galvatron11.jinkit.com"
ESXI_USER="root"
ESXI_AUTH="/etc/sushy/auth.conf"

[bjozsa@itamae ~]$ virsh -c esx://$ESXI_USER@$ESXI_HOST?no_verify=1\&authfile=$ESXI_AUTH list --all

 Id    Name                              State
---------------------------------------------------
 19    vCLS (2)                          running
 70    ai.jinkit.com                     running
 114   code.jinkit.com                   running
 117   mgmt-8xtdg-master-0               running
 118   mgmt-8xtdg-master-1               running
 119   mgmt-8xtdg-master-2               running
 120   mgmt-8xtdg-worker-whnqz           running
 121   mgmt-8xtdg-worker-bqnbm           running
 122   mgmt-8xtdg-worker-scf6s           running
 -     aio.test.cluster.telco.ocp.run    shut off
 -     dummy                             shut off
 -     hive-ai-test-01                   shut off
 -     kube-bgp-testing                  shut off
 -     lb01.jinkit.com                   shut off
 -     master01.mgmt.ztp.telco.ocp.run   shut off
 -     master02.mgmt.ztp.telco.ocp.run   shut off
 -     master03.mgmt.ztp.telco.ocp.run   shut off
 -     mgmt01.jinkit.com                 shut off
 -     splunk.jinkit.com                 shut off
 -     titanic                           shut off

[bjozsa@itamae ~]$

The auth.conf file should look like the following.

[credentials-esx]
username=root
password=password

[auth-esx-default]
credentials=esx

(Note: More information can be found within the Libvirt documentation)

Now that you've seen how this is constructed, it's easy to put this to use with a podman container.

ESXI_AUTH="/etc/sushy/auth.conf"

sudo podman create --net host --privileged --name sushy-vmware -v "/etc/sushy":/etc/sushy "${SUSHY_TOOLS_IMAGE}" sushy-emulator -i :: -p 8001 --config $ESXI_AUTH

Part V: Running via Systemd

Podman has been growing on me lately, and I mean by a lot! One feature I really love about Podman is the ability to easily create a systemd unit by issuing a simple command. We're going to use this to run sushy-emulator which will be instructed to restart when the host restarts.

# First, use Podman to create a systemd unit
sudo sh -c 'podman generate systemd --restart-policy=always -t 1 sushy-emulator > /etc/systemd/system/sushy-emulator.service'
sudo systemctl daemon-reload

# Next, use systemd to start and enable the Sushy-Emulator
sudo systemctl restart sushy-emulator.service
sudo systemctl enable sushy-emulator.service
sudo systemctl status sushy-emulator.service

What you're left with, is a systemd unit that will look something like this:

[bjozsa@itamae libvirt]$ sudo cat /etc/systemd/system/sushy-emulator.service
# container-1fa48994a2ce414fcaa5fc43bf73284248b98f5b5862bd96237eac6a8a442a56.service
# autogenerated by Podman 3.2.3
# Mon Aug 16 13:26:55 EDT 2021

[Unit]
Description=Podman container-1fa48994a2ce414fcaa5fc43bf73284248b98f5b5862bd96237eac6a8a442a56.service
Documentation=man:podman-generate-systemd(1)
Wants=network.target
After=network-online.target
RequiresMountsFor=/run/containers/storage

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=always
TimeoutStopSec=61
ExecStart=/usr/bin/podman start 1fa48994a2ce414fcaa5fc43bf73284248b98f5b5862bd96237eac6a8a442a56
ExecStop=/usr/bin/podman stop -t 1 1fa48994a2ce414fcaa5fc43bf73284248b98f5b5862bd96237eac6a8a442a56
ExecStopPost=/usr/bin/podman stop -t 1 1fa48994a2ce414fcaa5fc43bf73284248b98f5b5862bd96237eac6a8a442a56
PIDFile=/run/containers/storage/overlay-containers/1fa48994a2ce414fcaa5fc43bf73284248b98f5b5862bd96237eac6a8a442a56/userdata/conmon.pid
Type=forking

[Install]
WantedBy=multi-user.target default.target
[bjozsa@itamae libvirt]$

Part VI: Configuring the Firewall

Next, if you're running firewalld, make sure it is configured to allow remote connections to manage sushy-emulated hosts. You can check the status of firewalld, by running systemctl status firewalld:

# Create a firewall rule to allow advertizement of the Redfish API
sudo systemctl start firewalld
sudo firewall-cmd --add-port=8000/tcp

Part VII: Verify the Installation

There are a few tools that can manage Redfish-enabled hosts. Sushy-emulator is just one type of Redfish implementation. It's extremely useful for testing, or for managing virtual machines via common Redfish APIs.

To test your deployment, run the following command (using curl):

[bjozsa@itamae libvirt]$ curl http://192.168.3.99:8000/redfish/v1/Managers
{
    "@odata.type": "#ManagerCollection.ManagerCollection",
    "Name": "Manager Collection",
    "Members@odata.count": 2,
    "Members": [

          {
              "@odata.id": "/redfish/v1/Managers/71cb2e89-0fcf-43b3-a0d2-a91f594a7fde"
          },

          {
              "@odata.id": "/redfish/v1/Managers/928c9a76-ae0e-4c8b-9505-3c2155a3eb3f"
          }

    ],
    "Oem": {},
    "@odata.context": "/redfish/v1/$metadata#ManagerCollection.ManagerCollection",
    "@odata.id": "/redfish/v1/Managers",
    "@Redfish.Copyright": "Copyright 2014-2017 Distributed Management Task Force, Inc. (DMTF). For the full DMTF copyright policy, see http://www.dmtf.org/about/policies/copyright."
}[bjozsa@itamae libvirt]$

If you reviewed the output above, you will notice that there are 2 (two) systems running on the server itamae.

• 71cb2e89-0fcf-43b3-a0d2-a91f594a7fde
• 928c9a76-ae0e-4c8b-9505-3c2155a3eb3f

These ID's are the UUID of each system. They can either be auto-generated once you have created the virtual machine (similar to a previous article I wrote), or they can be created predictably. But this is something that can be saved for another article!

For now, you may want to consider some clients that can be leveraged for managing systems on Redfish-enabled hosts. Check out some of these options:

• Redfish System Management via Curl
• Redfish System Management via Redfishtool
• Redfish System Management via SushyCLI (Deprecated)