Calico Installation: OpenShift (VMware IPI)

Author: Brandon B. Jozsa

Note: The documentation on Calico's website is very well-written. This is not meant as a replacement or secondary source of truth to their own docs, but rather a blog about installation, use cases, and meant to go deeper with explanations than the upstream Calico documentation. As always, refer to the vendor documentation (both Calico and OpenShift) for any information and updates.

Pre-Installation

In general, the instructions from Calico's own Documentation can be followed, but let me expand on this a bit for our given installation use case: VMWare IPI. What you will need to collect are the following things prior to exploring the installation manifest below (install-config.yaml).

Sample install-config.yaml (provided below)
machineNetwork CIDR: 192.168.124.0/22
serviceNetwork CIDR: 10.96.0.0/12 (default for kubeadm as well)
clusterNetwork CIDR: 10.25.0.0/16 (more on this below)
$VCENTER_USER: Username for the vCenter Environment
$VCENTER_PASS: Password for the vCenter Environment

Now, you can do the same if you follow Calico's instructions by issuing the command openshift-install create install-config, but I've provided a final (and working) example for you below. You will want to explore the rest of the configuration file as well.

Example install-config.yaml:

apiVersion: v1
baseDomain: jinkit.com
compute:
- architecture: amd64
  hyperthreading: Enabled
  name: worker
  platform:
    vsphere:
      cpus: 12
      coresPerSocket: 2
      memoryMB: 36864
      osDisk:
        diskSizeGB: 120
  replicas: 3
controlPlane:
  architecture: amd64
  hyperthreading: Enabled
  name: master
  platform:
    vsphere:
      cpus: 8
      coresPerSocket: 2
      memoryMB: 24576
      osDisk:
        diskSizeGB: 120
  replicas: 3
metadata:
  creationTimestamp: null
  name: vcenter
networking:
  clusterNetwork:
  - cidr: 10.25.0.0/16
    hostPrefix: 23
  machineNetwork:
  - cidr: 192.168.124.0/22
  networkType: Calico
  serviceNetwork:
  - 10.96.0.0/12
platform:
  vsphere:
    apiVIP: 192.168.5.230
    cluster: openshift
    datacenter: JinkIT
    defaultDatastore: datastore11-01
    ingressVIP: 192.168.5.231
    network: VLAN4
    password: $VCENTER_PASS
    username: $VCENTER_USER
    vCenter: vcenter.jinkit.com
publish: External
pullSecret: $PULL_SECRET
sshKey: |
  ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDE1F7Fz3MGgOzst9h/2+5/pbeqCfFFhLfaS0Iu4Bhsr7RenaTdzVpbT+9WpSrrjdxDK9P3KProPwY2njgItOEgfJO6MnRLE9dQDzOUIQ8caIH7olzxy60dblonP5A82EuVUnZ0IGmAWSzUWsKef793tWjlRxl27eS1Bn8zbiI+m91Q8ypkLYSB9MMxQehupfzNzJpjVfA5dncZ2S7C8TFIPFtwBe9ITEb+w2phWvAE0SRjU3rLXwCOWHT+7NRwkFfhK/moalPGDIyMjATPOJrtKKQtzSdyHeh9WyKOjJu8tXiM/4jFpOYmg/aMJeGrO/9fdxPe+zPismC/FaLuv0OACgJ5b13tIfwD02OfB2J4+qXtTz2geJVirxzkoo/6cKtblcN/JjrYjwhfXR/dTehY59srgmQ5V1hzbUx1e4lMs+yZ78Xrf2QO+7BikKJsy4CDHqvRdcLlpRq1pe3R9oODRdoFZhkKWywFCpi52ioR4CVbc/tCewzMzNSKZ/3P0OItBi5IA5ex23dEVO/Mz1uyPrjgVx/U2N8J6yo9OOzX/Gftv/e3RKwGIUPpqZpzIUH/NOdeTtpoSIaL5t8Ki8d3eZuiLZJY5gan7tKUWDAL0JvJK+EEzs1YziBh91Dx1Yit0YeD+ztq/jOl0S8d0G3Q9BhwklILT6PuBI2nAEOS0Q== bjozsa@redhat.com

I will provide detailed instructions on how to set up vCenter and IPI in another blog post. For now, let's focus on the Calico installation and dive into some use cases.

Installation

Now that you're installation config is verified, let's begin with the installation. What we're going to do is create an installation directory, move our newly generated install-config.yaml into this directory, generate installation manifests, download the additional Calico manifests, and deploy the cluster declaritively (which is an OpenShift philosophy).

Create the installation directory and CD into it:

mkdir openshift-tigera-install && cd openshift-tigera-install

Move your install-config.yaml into the openshift-tigera-install directory, and generate installation manifests:

openshift-install create manifests

Download the Calico manifest artifacts:

curl https://docs.projectcalico.org/manifests/ocp/crds/01-crd-installation.yaml -o manifests/01-crd-installation.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/01-crd-imageset.yaml -o manifests/01-crd-imageset.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/01-crd-tigerastatus.yaml -o manifests/01-crd-tigerastatus.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_bgpconfigurations.yaml -o manifests/crd.projectcalico.org_bgpconfigurations.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_bgppeers.yaml -o manifests/crd.projectcalico.org_bgppeers.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_blockaffinities.yaml -o manifests/crd.projectcalico.org_blockaffinities.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_clusterinformations.yaml -o manifests/crd.projectcalico.org_clusterinformations.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_felixconfigurations.yaml -o manifests/crd.projectcalico.org_felixconfigurations.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_globalnetworkpolicies.yaml -o manifests/crd.projectcalico.org_globalnetworkpolicies.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_globalnetworksets.yaml -o manifests/crd.projectcalico.org_globalnetworksets.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_hostendpoints.yaml -o manifests/crd.projectcalico.org_hostendpoints.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_ipamblocks.yaml -o manifests/crd.projectcalico.org_ipamblocks.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_ipamconfigs.yaml -o manifests/crd.projectcalico.org_ipamconfigs.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_ipamhandles.yaml -o manifests/crd.projectcalico.org_ipamhandles.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_ippools.yaml -o manifests/crd.projectcalico.org_ippools.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_kubecontrollersconfigurations.yaml -o manifests/crd.projectcalico.org_kubecontrollersconfigurations.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_networkpolicies.yaml -o manifests/crd.projectcalico.org_networkpolicies.yaml\
curl https://docs.projectcalico.org/manifests/ocp/crds/calico/kdd/crd.projectcalico.org_networksets.yaml -o manifests/crd.projectcalico.org_networksets.yaml\
curl https://docs.projectcalico.org/manifests/ocp/tigera-operator/00-namespace-tigera-operator.yaml -o manifests/00-namespace-tigera-operator.yaml\
curl https://docs.projectcalico.org/manifests/ocp/tigera-operator/02-rolebinding-tigera-operator.yaml -o manifests/02-rolebinding-tigera-operator.yaml\
curl https://docs.projectcalico.org/manifests/ocp/tigera-operator/02-role-tigera-operator.yaml -o manifests/02-role-tigera-operator.yaml\
curl https://docs.projectcalico.org/manifests/ocp/tigera-operator/02-serviceaccount-tigera-operator.yaml -o manifests/02-serviceaccount-tigera-operator.yaml\
curl https://docs.projectcalico.org/manifests/ocp/tigera-operator/02-configmap-calico-resources.yaml -o manifests/02-configmap-calico-resources.yaml\
curl https://docs.projectcalico.org/manifests/ocp/tigera-operator/02-tigera-operator.yaml -o manifests/02-tigera-operator.yaml\
curl https://docs.projectcalico.org/manifests/ocp/01-cr-installation.yaml -o manifests/01-cr-installation.yaml

Make any appropriate changes (more on this later), and create cluster:

openshift-install create cluster --log-level=debug

Post-Installation (Optional)

After about 30 minutes or so, depending on your environment and several other factors, you will want to download and start using the calicoctl client. This will allow you to manage Calico, which we will get into a little later.

If you haven't left the installation directory, be sure to export your kubeconfig:

export KUBECONFIG=$(pwd)/auth/kubeconfig

Integration with OpenShifts OLM (Operator Lifecycle Manager)

I'll cover what the OLM does in a future article, but if you understand what OLM provides, and which to have the Calico Operator managed by the OLM, apply the following manifest:

oc apply -f - <<EOF\
apiVersion: operators.coreos.com/v1\
kind: OperatorGroup\
metadata:\
  name: tigera-operator\
  namespace: tigera-operator\
spec:\
  targetNamespaces:\
    - tigera-operator\
EOF

Next, create a subscription for the Calico Operator:

oc apply -f - <<EOF\
apiVersion: operators.coreos.com/v1alpha1\
kind: Subscription\
metadata:\
  name: tigera-operator\
  namespace: tigera-operator\
spec:\
  channel: stable\
  installPlanApproval: Manual\
  name: tigera-operator\
  source: certified-operators\
  sourceNamespace: openshift-marketplace\
  startingCSV: tigera-operator.v1.15.1\
EOF

Calico Management via the `calicoctl` Utility

I'm sure you've already heard about the nice things that Calico offers, in terms of being able to peer with other BGP peering points, leverage BGP route reflection, and utilize Calico's enhanced NetworkPolicies (and GlobalNetworkPolicies). We'll get deeper into these things in the next article, but for now you will want to install the calicoctl command-line utility. This utility will allow you to manage Calico-specific operations, unlike Tigera's paid version of Calico Enterprise which can be managed directly via Kubernetes/OpenShift constructs.

To deploy the calicoctl client, you can utilized the Kubernetes/OpenShift deployment right in the cluster. Apply the following manifest to get started:

oc apply -f https://docs.projectcalico.org/manifests/calicoctl.yaml

Alias the deployment for local use:

alias calicoctl="kubectl exec -i -n kube-system calicoctl -- /calicoctl"

Now test some some calicoctl commands. Let's start by getting a list of Calico managed nodes within the cluster.

calicoctl get nodes

Sample example:

NAME
vcenter-ztxns-master-0
vcenter-ztxns-master-1
vcenter-ztxns-master-2
vcenter-ztxns-worker-mrqcm
vcenter-ztxns-worker-mrqq4
vcenter-ztxns-worker-rtq54

Next, have a look at the Calico managed ippools. You'll noticed that this original IP Pool was assigned automatically via the manifest we used in our example above.

calicoctl get ippools

Sample example:

NAME                  CIDR           SELECTOR
default-ipv4-ippool   10.25.0.0/16   all()

Conclusion

It's safe to say that Calico is easy to install as an alternative to the OpenShift-SDN (if this is your preference). In the next section, we'll compare some advanced configuration scenarios like how to implement BGP Peering, how Calico Network Policy Model works, as well as cover some alternatives. I'll also cover some Calico implementation considerations that you'll be faced with when implementing Calico; you know, the things that people won't initially tell you about. So stick around, and make sure to subscribe to follow this series. It'll be a good one!

Pre-Installation

Installation

Post-Installation (Optional)

Integration with OpenShifts OLM (Operator Lifecycle Manager)

Calico Management via the calicoctl Utility

Conclusion

Calico Management via the `calicoctl` Utility