Service Unreachable
Trace the request hop by hop so you know whether DNS, the Service, endpoints, ports, policy or the application is failing.
What does “Service unreachable” mean?
It can mean several different things: the name does not resolve, the Service IP cannot be reached, the Service has no ready backends, traffic is blocked, the target port is wrong, or the application responds incorrectly. First locate the broken hop.
The traffic path
Step 1 — Inspect the Service
kubectl get svc <service> -n <namespace>
kubectl describe svc <service> -n <namespace>Check the Service type, ClusterIP, selector, client-facing port and backend targetPort.
port. Kubernetes forwards that traffic to the backend targetPort. If the application listens on 8080 but targetPort points to 8000, the Service can exist normally while requests fail.Step 2 — Does the Service have endpoints?
kubectl get endpointslice -n <namespace> -l kubernetes.io/service-name=<service>NAME ADDRESSTYPE PORTS ENDPOINTS AGE
payments-7fd2 IPv4 8080 10.244.2.17 4hWhat to check: an endpoint address and expected backend port should be present. No suitable endpoint is a strong reason to compare Service selectors, pod labels and readiness before debugging DNS or HTTP. If no suitable backend endpoints appear, compare Service selectors with pod labels:
kubectl get svc <service> -n <namespace> -o yaml
kubectl get pods -n <namespace> --show-labelsAlso confirm backend pods are Ready. Endpoint readiness matters because an unhealthy backend should not receive normal Service traffic.
Step 3 — Test from the client side
Use an existing authorized diagnostic/client pod where possible and test the same destination the failing workload uses. A typical HTTP test might be:
curl -v http://<service>:<port>/healthInterpret the failure: “could not resolve host” points toward DNS; “connection refused” suggests something reachable is not accepting the connection; timeout suggests routing/policy/network path; an HTTP 4xx/5xx means connectivity exists and the application/proxy layer is responding.
Step 4 — If DNS fails
Try the Kubernetes service FQDN:
<service>.<namespace>.svc.cluster.localThen inspect cluster DNS components and configuration according to your distribution. If direct Service IP access works while name resolution fails, DNS is a much stronger lead than Service selectors.
Step 5 — If endpoints exist but traffic fails
Verify the application actually listens on the expected port and an interface reachable through the pod network—not only 127.0.0.1. Check application logs and readiness behavior.
Common HTTP status codes
2xx — tested endpoint returned success; compare the original client path, headers and auth.
301/302/307/308 — redirect; check scheme, hostname, path and proxy/Ingress behavior.
400 — invalid request; inspect body, headers and rewriting.
401 / 403 — authentication or authorization path.
404 — route/resource not found; check URL, app routing and proxy/Ingress rules.
405 — HTTP method not allowed.
429 — rate limiting/throttling.
500 — responder hit an internal error.
502 — proxy/gateway could not get a valid upstream response.
503 — unavailable/no healthy upstream depending on the responder; check readiness, endpoints and backend availability.
504 — gateway timed out waiting for upstream.
Compare Service and backend
curl -v http://<service>:<port>/health
curl -v http://<pod-ip>:<target-port>/healthDirect Pod-IP access is diagnostic only; pod addresses are ephemeral.
Cross-namespace DNS
payments
payments.backend
payments.backend.svc.cluster.localStep 6 — Check NetworkPolicies
kubectl get networkpolicy -A
kubectl describe networkpolicy <policy> -n <namespace>Check both ingress to the backend and egress from the client where policies apply. Verify pod/namespace selectors and allowed ports.
Do not delete production NetworkPolicies broadly just to “see if it works.” Prefer a controlled policy change consistent with your source of truth and security model.
Know when to leave the Service layer
If the Service works internally but an external hostname fails, continue with Ingress/Gateway routes, load balancer, TLS, external DNS and firewall/security controls.
Verify
Retest from the original client path. Confirm DNS resolution if used, Service connection, backend response and application-level success.
Common mistakes
- Debugging DNS when the Service has zero endpoints.
- Confusing Service
portwithtargetPort. - Testing only from a laptop instead of the failing workload path.
- Removing all NetworkPolicies as the first test.
- Assuming a TCP connection proves the application is healthy.