kubectl debug - Interactive Kubernetes Debugging

Expert knowledge for debugging Kubernetes resources using kubectl debug

• ephemeral containers, pod copies, and node access.

Core Capabilities

kubectl debug automates common debugging tasks:

• Ephemeral Containers: Add debug containers to running pods without restart

• Pod Copying: Create modified copies for debugging (different images, commands)

• Node Debugging: Access node host namespaces and filesystem

Context Safety (CRITICAL)

Always specify --context explicitly in every kubectl command:

CORRECT: Explicit context

kubectl --context=prod-cluster debug mypod -it --image=busybox

WRONG: Relying on current context

kubectl debug mypod -it --image=busybox # Which cluster?

Quick Reference

Add Ephemeral Debug Container

Interactive debugging with busybox

kubectl --context=my-context debug mypod -it --image=busybox

Target specific container's process namespace

kubectl --context=my-context debug mypod -it --image=busybox --target=mycontainer

Use a specific debug profile

kubectl --context=my-context debug mypod -it --image=busybox --profile=netadmin

Copy Pod for Debugging

Create debug copy

kubectl --context=my-context debug mypod -it --copy-to=mypod-debug --image=busybox

Copy and change container image

kubectl --context=my-context debug mypod --copy-to=mypod-debug --set-image=app=busybox

Copy and modify command

kubectl --context=my-context debug mypod -it --copy-to=mypod-debug --container=myapp -- sh

Copy on same node

kubectl --context=my-context debug mypod -it --copy-to=mypod-debug --same-node --image=busybox

Debug Node

Interactive node debugging (host namespaces, filesystem at /host)

kubectl --context=my-context debug node/mynode -it --image=busybox

With sysadmin profile for full capabilities

kubectl --context=my-context debug node/mynode -it --image=ubuntu --profile=sysadmin

Debug Profiles

Profile Use Case Capabilities

legacy

Default, unrestricted Full access (backwards compatible)

general

General purpose Moderate restrictions

baseline

Minimal restrictions Pod security baseline

netadmin

Network troubleshooting NET_ADMIN capability

restricted

High security environments Strictest restrictions

sysadmin

System administration SYS_PTRACE, SYS_ADMIN

Network debugging (tcpdump, netstat, ss)

kubectl --context=my-context debug mypod -it --image=nicolaka/netshoot --profile=netadmin

System debugging (strace, perf)

kubectl --context=my-context debug mypod -it --image=ubuntu --profile=sysadmin

Common Debug Images

Image Size Use Case

busybox

~1MB Basic shell, common utilities

alpine

~5MB Shell with apk package manager

ubuntu

~77MB Full Linux with apt

nicolaka/netshoot

~350MB Network debugging (tcpdump, dig, curl, netstat)

gcr.io/k8s-debug/debug

Varies Official Kubernetes debug image

Debugging Patterns

Network Connectivity Issues

Add netshoot container for network debugging

kubectl --context=my-context debug mypod -it
--image=nicolaka/netshoot
--profile=netadmin

Inside container:

- tcpdump -i any port 80

- dig kubernetes.default

- curl -v http://service:port

- ss -tlnp

- netstat -an

Application Crashes

Copy pod with different entrypoint to inspect

kubectl --context=my-context debug mypod -it
--copy-to=mypod-debug
--container=app
-- sh

Inside: check filesystem, env vars, config files

Process Inspection

Target container's process namespace

kubectl --context=my-context debug mypod -it
--image=busybox
--target=mycontainer

Inside: ps aux, /proc inspection

Node-Level Issues

Debug node with host access

kubectl --context=my-context debug node/worker-1 -it
--image=ubuntu
--profile=sysadmin

Inside:

- Host filesystem at /host

- chroot /host for full access

- journalctl, systemctl, dmesg

Non-Destructive Debugging

Create copy, keeping original running

kubectl --context=my-context debug mypod -it
--copy-to=mypod-debug
--same-node
--share-processes
--image=busybox

Original pod continues serving traffic

Debug copy shares storage if on same node

Key Options

Option Description

-it

Interactive TTY (required for shell access)

--image

Debug container image

--container

Name for the debug container

--target

Share process namespace with this container

--copy-to

Create a copy instead of ephemeral container

--same-node

Schedule copy on same node (with --copy-to )

--set-image

Change container images in copy

--profile

Security profile (legacy, netadmin, sysadmin, etc.)

--share-processes

Enable process namespace sharing (default: true with --copy-to)

--replace

Delete original pod when creating copy

Best Practices

• Use appropriate profiles - Match capabilities to debugging needs

• Prefer ephemeral containers - Less disruptive than pod copies

• Use --copy-to for invasive debugging - Preserve original pod

• Clean up debug pods - Delete copies after debugging

• Use --same-node

• For accessing shared storage/network conditions

Cleanup

List debug pod copies

kubectl --context=my-context get pods | grep -E "debug|copy"

Delete debug pods

kubectl --context=my-context delete pod mypod-debug

Requirements

• Kubernetes 1.23+ for ephemeral containers (stable)

• Kubernetes 1.25+ for debug profiles

• RBAC permissions for pods/ephemeralcontainers

For detailed option reference, examples, and troubleshooting patterns, see REFERENCE.md.