ImagePullBackOff

KUBERNETES RUNBOOK

ImagePullBackOff

Find out why a node cannot retrieve your container image and fix the correct layer: image reference, registry access, credentials or network.

What does ImagePullBackOff mean?

Kubernetes asked the node to pull an image, the pull failed, and kubelet is waiting before retrying. You may see ErrImagePull first and ImagePullBackOff after repeated failures.

60-second diagnosis

kubectl describe pod <pod> -n <namespace>

Go directly to Events. The message usually tells you which branch to follow.

Image pull fails↓Read Events↓
not found / manifest unknown → image or tag
unauthorized / denied → credentials or permissions
timeout / DNS → node-to-registry network
rate limit → registry policy/quota
x509 / TLS → trust or certificate path

Step 1 — Read and interpret Events

Example:

Failed to pull image "registry.example.com/team/api:v3":
rpc error: ... not found

This tells you the node reached the registry but the requested artifact could not be found. Do not start by recreating registry credentials.

Step 2 — Verify the exact image

kubectl get pod <pod> -n <namespace> -o jsonpath="{.spec.containers[*].image}"
registry.example.com/team/api:v3

What to check: compare this exact registry/repository/tag or digest with the artifact that CI/CD actually published. Check registry hostname, repository path, image name, tag or digest. Compare it with the artifact your CI/CD process actually pushed.

Step 3 — If Events say unauthorized

Check which pull secret the pod references:

kubectl get pod <pod> -n <namespace> -o jsonpath="{.spec.imagePullSecrets[*].name}"
kubectl get serviceaccount <service-account> -n <namespace> -o yaml
kubectl get secret <pull-secret> -n <namespace>

The secret must be in the same namespace as the pod and contain credentials authorized for that repository. Do not paste decoded credentials into tickets or chat.

Step 4 — If Events show timeout, DNS or TLS errors

The image is pulled by the node/container runtime, so application-pod networking tests alone may not reproduce the path. Identify the node:

kubectl get pod <pod> -n <namespace> -o wide

If only one node/pool fails, compare node DNS, proxy, egress/firewall rules, registry reachability, container-runtime configuration and certificate trust.

Step 5 — Understand imagePullPolicy

Always asks the runtime to resolve/pull according to policy on container start; IfNotPresent can use a locally present image. Changing policy to bypass a broken release or registry path hides the problem rather than fixing it.

When you correct an image reference, pull secret or workload setting, make the change in the Deployment/StatefulSet manifest or your Git/Helm source of truth, then roll it out normally.

Verify

kubectl get pod <pod> -n <namespace> -w
kubectl describe pod <pod> -n <namespace>

Look for a successful pull followed by container creation/start. If the image now pulls but the container crashes, continue with the CrashLoopBackOff runbook.

Common mistakes

  • Starting with kubectl logs. If the image never pulled, the application container may never have started. Start with pod Events.
  • Assuming every pull failure is a credential problem. Read the exact event first.
  • Checking only the intended Deployment value. Verify the image reference on the actual Pod.
  • Replacing imagePullSecrets when the error says the image/tag is missing.
  • Forgetting imagePullSecrets are namespace-scoped.
  • Using a successful laptop pull as proof cluster nodes can reach the registry.
  • Changing imagePullPolicy to mask the failure. A cached image can hide a broken registry/release path.