TCPDump for OpenShift Workloads

Networking Sep 13, 2021

Author: Brandon B. Jozsa

Today's post will be rather short, and it really came from the need to troubleshoot a specific pod within an OpenShift cluster. I ran into a couple of issues, but wanted to use this post as a single source (for anyone else running into problems).

My ultimate goal is to have a client endpoint launch a TCPDump container either into a pod (as a sidecar) or by using the new Fennec Project that a few of us have been collaborating on recently. I will update this article as I work through each solution.

Project: ksniff

ksniff is a project that allows admins to tcpdump within a pod by deploying a sidecar and sniffing traffic in real-time. This can be either captured locally for review later, or it can be captured and viewed in NRT (near real-time). But before we can use ksniff, it will need to be installed first.

Installation

As a kubectl plugin, the most popular way to install this by using krew. It's really simple to install with the following command.

# Make sure that git is installed first, and run the following:
(
  set -x; cd "$(mktemp -d)" &&
  OS="$(uname | tr '[:upper:]' '[:lower:]')" &&
  ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&
  curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/krew.tar.gz" &&
  tar zxvf krew.tar.gz &&
  KREW=./krew-"${OS}_${ARCH}" &&
  "$KREW" install krew
)

# Next, add the following to your path:
export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"

(Note: the command above will work for both bash and zsh)

Now that krew is available for both kubectl and oc commands, use the following to install ksniff.

kubectl krew install sniff
Installing for kubectl
oc krew install sniff
Installing for oc

Next, you will want to install Wireshark, as describe in the following link HERE.

Usage

Armed with sniff/ksniff, you can start having some fun. In order to run a colorized tcpdump of traffic (similar to what you'd see in Wireshark), run the following command.

WORKLOAD="productpage-v1-658849bb5-pcwsb"
NAMESPACE="bookinfo"

oc sniff $WORKLOAD -p -n $NAMESPACE --socket /var/run/crio/crio.sock -o - | wireshark --color -r -

This will produce output that's rich in detail and colorized, as with the example below.

Tags