manishpawar1/msp_power_troubleshooting repository overview

⁠msp_power_troubleshooting

UBI 9 based network and diagnostic toolbox for troubleshooting Kubernetes pods. Use it as a sidecar, an ephemeral debug container, or with plain docker run. All tools are pre-installed, so the container needs no internet access at start and is ready immediately.

⁠Tags and platforms

Platforms: linux/amd64, linux/arm64. Lightweight: about 380 MB on disk, no JDK (see the tag page for the compressed size).

⁠Quick start

⁠Kubernetes sidecar

The image idles by itself, so no command or args are needed:

containers:
  - name: power-troubleshooting
    image: manishpawar1/msp_power_troubleshooting:1.1.0
    imagePullPolicy: IfNotPresent
    resources:
      requests: { cpu: 100m, memory: 128Mi }
      limits:   { cpu: 500m, memory: 512Mi }

Open a shell in the sidecar:

kubectl exec -it -n <namespace> <pod> -c power-troubleshooting -- bash

⁠Ephemeral debug container (no redeploy)

kubectl debug -it -n <namespace> <pod> \
  --image=manishpawar1/msp_power_troubleshooting:1.1.0 \
  --target=<app-container> --profile=sysadmin -- bash

--target shares the app container's process namespace so ps and htop can see its processes, and files can be read via /proc/<pid>/root/.

⁠Plain Docker (laptop / VM)

docker run --rm -it manishpawar1/msp_power_troubleshooting:1.1.0 bash
docker run --rm manishpawar1/msp_power_troubleshooting:1.1.0 dig +short example.com
docker run --rm -it --cap-add NET_RAW --cap-add NET_ADMIN manishpawar1/msp_power_troubleshooting:1.1.0 tcpdump -D

If the repository is private, run docker login first (and use an imagePullSecrets entry in Kubernetes).

⁠Included tools

Not included: kubectl and a JDK (left out on purpose to keep the image small). Run JVM tools such as keytool, jcmd and jstack in the application container with kubectl exec -c <app-container> -- ....

⁠Common examples

# Is the app answering? Where does the time go?
curl -s -o /dev/null -w 'dns=%{time_namelookup} connect=%{time_connect} tls=%{time_appconnect} ttfb=%{time_starttransfer} total=%{time_total} code=%{http_code}\n' http://<service>:<port>/<path>

# JSON from an API
curl -s http://localhost:<port>/<path> | jq

# Port reachability and DNS
ncat -zv -w 3 <host> <port>
dig +short <service-name>

# Packet capture (rotating files, small snaplen)
tcpdump -i any -nn -s 128 -C 100 -W 5 -w /tmp/c.pcap port <port>

# TLS certificate details and chain verification
openssl s_client -connect <host>:<tls-port> -servername <host> </dev/null 2>/dev/null | openssl x509 -noout -subject -issuer -dates
openssl s_client -connect <host>:<tls-port> -servername <host> -CAfile /tmp/ca.pem </dev/null 2>&1 | grep -i 'verify return'

# Kerberos
KRB5_TRACE=/dev/stderr kinit -V <principal>
kinit -kt /path/to.keytab <principal> && klist -e
kvno <service-principal>

# LDAP bind test (prompts for the password)
ldapwhoami -x -H ldap://<ldap-host>:389 -D "<bind-dn>" -W

# Bandwidth between two pods
iperf3 -s                         # in pod A
iperf3 -c <pod-A-ip> -t 30 -P 4   # in pod B

# Signed S3-compatible request (set the variables first; do not paste secrets into shared logs)
curl -s --aws-sigv4 "aws:amz:${S3_REGION}:s3" --user "${S3_AK}:${S3_SK}" ${S3_ENDPOINT}/

⁠What the sidecar can see

A sidecar shares the pod's network namespace (same IP, localhost, ports, DNS), so it can test the application from the inside and capture its traffic with tcpdump.

It does not share processes or filesystem by default. To see the app container's processes and files, share the process namespace (for example kubectl debug --target=...) and add the SYS_PTRACE capability, or mount a shared volume.

⁠Security notes

• Runs as root; remove it when troubleshooting is done and use it only where needed.
• tcpdump needs NET_RAW; ethtool/mii-tool may need NET_ADMIN. Restricted PodSecurity profiles will block these.
• Packet captures can contain credentials and data. Capture narrowly, store securely and delete afterwards.
• Pin the exact tag or digest in deployments, not latest.
• Do not leave listeners (iperf3 -s, ncat -l, httpd) running.
• The image is rebuilt periodically and scanned with Trivy; the release had no fixable HIGH/CRITICAL findings. Some unfixed upstream OS CVEs remain in client/utility packages (for example bind client tools, ab/htpasswd, util-linux, openssl, pcre2, python/pip, expat, vim); none run as servers in the container.

⁠Build details

• Base: registry.access.redhat.com/ubi9/ubi (RHEL 9 userland, glibc).
• Packages are installed at build time from UBI 9, EPEL 9 and CentOS Stream 9 (GPG-checked; extra repos are removed before the image is finalized).
• The container idles through its default CMD (a sleep loop that exits cleanly on SIGTERM), so any command you pass at run time overrides it.
• Python is 3.9 (RHEL 9); create a venv for pip install.

⁠Known limitations

• Root user, troubleshooting use only.
• No kubectl.
• No JDK: run keytool, jcmd, jstack and similar tools in the application container.
• Kerberos needs your krb5.conf (mount it or set KRB5_CONFIG) and a reachable KDC; keytabs and credential caches must be provided at run time and never baked into the image.
• Python 3.9 may be too old for some current PyPI packages.
• Anonymous Docker Hub pulls are rate-limited per IP; use imagePullPolicy: IfNotPresent, a pull secret or a registry mirror for busy clusters.

⁠Updates and versioning

Rebuilt periodically and when a relevant fixed HIGH/CRITICAL CVE is published. Semantic versioning: patch (1.1.x) for security rebuilds, minor (1.x.0) for tool changes, major for breaking changes. Release tags are not overwritten; 1.1 and latest move.

⁠Disclaimer

Personal troubleshooting image, provided as is, without warranty. Red Hat and UBI are trademarks of Red Hat, Inc.