# Cert Manager Troubleshooting

> Use when TLS certificates are not being issued or renewed, when Certificate or CertificateRequest resources show errors, when ACME challenges fail, or when Issuer or ClusterIssuer resources are not ready

- **Type:** Skill
- **Install:** `agentstack add skill-foxj77-claude-code-skills-cert-manager-troubleshooting`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [foxj77](https://agentstack.voostack.com/s/foxj77)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [foxj77](https://github.com/foxj77)
- **Source:** https://github.com/foxj77/claude-code-skills/tree/main/cert-manager-troubleshooting

## Install

```sh
agentstack add skill-foxj77-claude-code-skills-cert-manager-troubleshooting
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# cert-manager Troubleshooting

Diagnose failures in cert-manager — the controller that automates TLS certificate issuance and renewal from sources like Let's Encrypt (ACME), HashiCorp Vault, Venafi, and self-signed CAs.

## Keywords

cert-manager, certificate, tls, ssl, https, letsencrypt, acme, issuer, clusterissuer, certificaterequest, order, challenge, http01, dns01, self-signed, ca, renew, renewal, expired, not-ready, ingress, annotation

## When to Use This Skill

- Certificate resources show `Ready: False` or `Issuing` for too long
- CertificateRequest is stuck or shows errors
- ACME challenges (HTTP-01 or DNS-01) are failing
- Issuer or ClusterIssuer shows `Ready: False`
- TLS secrets are not being created or contain expired certificates
- Ingress TLS annotations are not triggering certificate creation
- Certificate renewal is not happening before expiry

### When NOT to Use

- DNS records are not being created (even if needed for DNS-01 challenges) → use [external-dns-troubleshooting](../external-dns-troubleshooting)
- Issuer credentials come from an external store and the store is failing → use [external-secrets-troubleshooting](../external-secrets-troubleshooting)
- cert-manager resources are delivered by Flux and not appearing → use [flux-troubleshooting](../flux-troubleshooting)

## Related Skills

- [external-dns-troubleshooting](../external-dns-troubleshooting) - DNS records required for DNS-01 challenges
- [external-secrets-troubleshooting](../external-secrets-troubleshooting) - Issuer credentials from external stores
- [kyverno-troubleshooting](../kyverno-troubleshooting) - Policies may block cert-manager resources
- [k8s-namespace-troubleshooting](../k8s-namespace-troubleshooting) - General namespace diagnosis
- [k8s-platform-operations](../k8s-platform-operations) - Cluster-wide operations
- [flux-troubleshooting](../flux-troubleshooting) - GitOps delivery of cert-manager resources

## Quick Reference

| Task | Command |
|------|---------|
| Check cert-manager pods | `kubectl get pods -n cert-manager` |
| List Certificates | `kubectl get certificate -A` |
| List Issuers | `kubectl get issuer -A && kubectl get clusterissuer` |
| Check CertificateRequests | `kubectl get certificaterequest -A` |
| Check Orders (ACME) | `kubectl get order -A` |
| Check Challenges (ACME) | `kubectl get challenge -A` |
| View cert-manager logs | `kubectl logs -n cert-manager deploy/cert-manager --tail=200` |
| Check cert expiry | `kubectl get secret ${SECRET} -n ${NS} -o jsonpath='{.data.tls\.crt}' \| base64 -d \| openssl x509 -noout -dates` |

---

## Diagnostic Workflow

```
TLS certificate not working?
├─ Certificate resource exists?
│   ├─ No → Check Ingress annotations or create Certificate (Section 4)
│   └─ Yes → Check Certificate status
│       ├─ Ready: True → Certificate issued, check if correct (Section 6)
│       └─ Ready: False → Check CertificateRequest
│           ├─ No CertificateRequest → Issuer reference broken (Section 2)
│           ├─ CertificateRequest pending → Check Issuer health (Section 2)
│           └─ CertificateRequest denied → Check approval/RBAC
├─ Using ACME (Let's Encrypt)?
│   ├─ Order created? → Check Order status
│   │   ├─ Order pending → Check Challenge status (Section 3)
│   │   ├─ Order invalid → All challenges failed
│   │   └─ Order errored → ACME account issue (Section 2)
│   └─ Challenge status?
│       ├─ HTTP-01 pending → Solver pod/ingress issues (Section 3)
│       └─ DNS-01 pending → DNS provider issues (Section 3)
└─ Certificate expired?
    └─ Renewal not triggered → Check renewal config (Section 5)
```

---

## Section 1: Controller Health

```bash
# Check all cert-manager components
kubectl get pods -n cert-manager -o wide

# Controller logs (main component)
kubectl logs -n cert-manager deploy/cert-manager --tail=200

# Webhook logs (validates/converts resources)
kubectl logs -n cert-manager deploy/cert-manager-webhook --tail=100

# CA injector logs (injects CA bundles)
kubectl logs -n cert-manager deploy/cert-manager-cainjector --tail=100

# Check CRDs
kubectl get crd | grep cert-manager

# Check webhook connectivity
kubectl get validatingwebhookconfigurations | grep cert-manager
kubectl get mutatingwebhookconfigurations | grep cert-manager
```

### Controller Issues

| Symptom | Cause | Diagnostic |
|---------|-------|-----------|
| Certificate stuck at "Issuing" | Controller not processing | Check controller logs for errors |
| Any cert-manager resource creation fails | Webhook unreachable | Check webhook pod, Service, and network policies |
| "conversion webhook" errors | CRD version mismatch | Check if cert-manager version matches CRD version |
| Leader election errors | Multiple controllers competing | Check for duplicate installations |

---

## Section 2: Issuer and ClusterIssuer

```bash
# Check Issuer status
kubectl get issuer -n ${NS} -o wide
kubectl describe issuer ${ISSUER_NAME} -n ${NS}

# Check ClusterIssuer status
kubectl get clusterissuer -o wide
kubectl describe clusterissuer ${ISSUER_NAME}

# Check the Certificate's issuer reference
kubectl get certificate ${CERT_NAME} -n ${NS} -o jsonpath='{.spec.issuerRef}'
```

### ACME Issuer (Let's Encrypt)

```bash
# Check ACME account registration
kubectl describe issuer ${ISSUER_NAME} -n ${NS} | grep -A10 "Status:"

# Check ACME account secret
kubectl get secret ${ISSUER_NAME}-account-key -n ${NS} 2>/dev/null

# Verify ACME server URL
kubectl get issuer ${ISSUER_NAME} -n ${NS} -o jsonpath='{.spec.acme.server}'
```

| ACME Server | URL | Use Case |
|-------------|-----|----------|
| Let's Encrypt Production | `https://acme-v02.api.letsencrypt.org/directory` | Production certs |
| Let's Encrypt Staging | `https://acme-staging-v02.api.letsencrypt.org/directory` | Testing (fake certs, higher rate limits) |

| Error | Cause | Diagnostic |
|-------|-------|-----------|
| "account not registered" | ACME registration failed | Check email config and account secret status |
| "invalid account" | Account key mismatch | Check if account key secret is corrupted or stale |
| Issuer `Ready: False` | Can't reach ACME server | Check network/proxy connectivity and verify server URL |

### CA Issuer

```bash
# Check CA secret exists and has required keys
kubectl get secret ${CA_SECRET} -n ${NS}
kubectl get secret ${CA_SECRET} -n ${NS} -o jsonpath='{.data}' | jq 'keys'
# Must contain: tls.crt, tls.key
```

### Vault Issuer

```bash
# Check Vault connection
kubectl describe issuer ${ISSUER_NAME} -n ${NS} | grep -A20 "Spec:"

# Logs for Vault errors
kubectl logs -n cert-manager deploy/cert-manager --tail=200 | grep -iE 'vault|pki|sign|auth'
```

---

## Section 3: ACME Challenges (HTTP-01 and DNS-01)

```bash
# List all challenges
kubectl get challenge -A -o wide

# Describe failing challenge
kubectl describe challenge ${CHALLENGE_NAME} -n ${NS}

# Check challenge state
kubectl get challenge -A -o json | jq -r '.items[] | "\(.metadata.namespace)/\(.metadata.name)\t\(.spec.type)\t\(.status.state)\t\(.status.reason // "no reason")"'
```

### HTTP-01 Challenge Troubleshooting

```bash
# Check solver pod
kubectl get pods -n ${NS} -l acme.cert-manager.io/http01-solver=true

# Check solver Ingress/Service
kubectl get ingress -n ${NS} -l acme.cert-manager.io/http01-solver=true
kubectl get svc -n ${NS} -l acme.cert-manager.io/http01-solver=true

# Test challenge URL locally
# The URL is: http:///.well-known/acme-challenge/
kubectl get challenge ${CHALLENGE_NAME} -n ${NS} -o jsonpath='{.spec.token}'
```

| HTTP-01 Problem | Cause | Diagnostic |
|-----------------|-------|-----------|
| Solver pod not created | RBAC or class annotation issue | Check cert-manager logs and verify ingress class |
| Solver pod pending | Resource constraints | Check node capacity and resource quotas |
| Challenge URL returns 404 | Ingress not routing to solver | Check ingress class, annotations, and controller |
| Challenge URL unreachable | Firewall blocks port 80 | Verify port 80 is open from internet to ingress controller |
| "wrong status code: 503" | Load balancer health check failing | Check LB health check config and solver pod readiness |
| Challenge stays pending | Let's Encrypt can't reach cluster | Verify domain resolves to cluster ingress IP |

### DNS-01 Challenge Troubleshooting

```bash
# Check which DNS provider is configured
kubectl get issuer ${ISSUER_NAME} -n ${NS} -o json | jq '.spec.acme.solvers[].dns01'

# Check for DNS provider credentials
kubectl get challenge ${CHALLENGE_NAME} -n ${NS} -o json | jq '.spec.solver.dns01'

# Verify TXT record was created
dig TXT _acme-challenge.${DOMAIN} @8.8.8.8

# Check cert-manager DNS provider logs
kubectl logs -n cert-manager deploy/cert-manager --tail=300 | grep -iE 'dns|challenge|present|cleanup|txt'
```

| DNS-01 Problem | Cause | Diagnostic |
|----------------|-------|-----------|
| TXT record not created | Provider auth failed | Check DNS provider credentials in Issuer |
| TXT record exists but challenge fails | DNS propagation delay | Check nameserver configuration and `--dns01-recursive-nameservers-only` setting |
| Wrong zone | DNS split-horizon or sub-domain issue | Check `--dns01-recursive-nameservers` setting |
| "zone not found" | Provider can't find the DNS zone | Verify zone exists and credentials have access |
| Cleanup failed (TXT not removed) | Provider API error on delete | Check for stale `_acme-challenge` TXT records at provider |

### Common DNS-01 Providers

| Provider | Secret Type | Key Fields |
|----------|-------------|------------|
| Route53 | AWS credentials or IRSA | `accessKeyID`, `secretAccessKeySecretRef`, or IRSA annotation |
| Azure DNS | Service principal or Workload Identity | `clientID`, `clientSecretSecretRef`, `subscriptionID`, `resourceGroupName` |
| CloudFlare | API token | `apiTokenSecretRef` |
| Google Cloud DNS | Service account key | `serviceAccountSecretRef`, `project` |

---

## Section 4: Certificate and Ingress Configuration

```bash
# Check Certificate spec
kubectl get certificate ${CERT_NAME} -n ${NS} -o yaml

# Check CertificateRequest chain
kubectl get certificaterequest -n ${NS} -o wide
kubectl describe certificaterequest ${CR_NAME} -n ${NS}

# Check if Ingress annotations trigger cert creation
kubectl get ingress -n ${NS} -o json | jq '.items[] | {name: .metadata.name, annotations: .metadata.annotations, tls: .spec.tls}'
```

### Ingress Annotation Issues

| Annotation | Purpose | Common Mistake |
|------------|---------|----------------|
| `cert-manager.io/issuer` | Namespace Issuer | Using ClusterIssuer name with this annotation |
| `cert-manager.io/cluster-issuer` | ClusterIssuer | Typo in name or using Issuer name |
| `cert-manager.io/issue-temporary-certificate` | Temp cert during issuance | Forgetting this causes TLS errors during initial issuance |
| `cert-manager.io/common-name` | Override CN | Ignored if dnsNames are set |

### Certificate Spec Problems

| Problem | Cause | Diagnostic |
|---------|-------|-----------|
| "issuer not found" | Wrong `issuerRef.name` or `issuerRef.kind` | Verify issuer exists and kind matches (Issuer vs ClusterIssuer) |
| "dnsNames required" | No domains specified | Check if `dnsNames` is present in Certificate spec |
| "secret already exists" | Conflict with manually created Secret | Check if a Secret with the same name already exists |
| CertificateRequest not created | Spec validation failed | Check cert-manager webhook logs |

---

## Section 5: Renewal and Expiry

```bash
# Check certificate dates
kubectl get certificate -A -o json | jq -r '.items[] | "\(.metadata.namespace)/\(.metadata.name)\t\(.status.notAfter)\t\(.status.renewalTime)\tReady=\(.status.conditions[0].status)"'

# Check from the actual TLS secret
kubectl get secret ${SECRET} -n ${NS} -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -dates -subject

# To force renewal, recommend to user:
# kubectl cert-manager renew ${CERT_NAME} -n ${NS}
```

| Problem | Cause | Diagnostic |
|---------|-------|-----------|
| Not renewing before expiry | `renewBefore` too short or controller stopped | Check `renewBefore` setting (default: 2/3 of cert lifetime) |
| Renewed but old cert in Secret | Secret not updated | Check if Secret was manually modified; recommend deleting Secret so cert-manager recreates it |
| Let's Encrypt rate limited | Too many issuances for same domain | Check `describe order` for rate limit messages; recommend using staging for testing |
| Wildcard cert not renewing | DNS-01 solver broken | Check DNS-01 challenge flow specifically |

### Let's Encrypt Rate Limits

| Limit | Value | Scope |
|-------|-------|-------|
| Certificates per domain | 50/week | Registered domain (e.g., example.com) |
| Duplicate certificates | 5/week | Same set of domain names |
| Failed validations | 5/hour | Per account, per hostname |
| New registrations | 10/3 hours | Per IP |

---

## Section 6: Verifying Issued Certificates

```bash
# Check certificate details from the K8s Secret
kubectl get secret ${SECRET} -n ${NS} -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -text | head -30

# Verify cert chain
kubectl get secret ${SECRET} -n ${NS} -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl verify -show_chain

# Check cert matches key
CERT_MOD=$(kubectl get secret ${SECRET} -n ${NS} -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -modulus | md5)
KEY_MOD=$(kubectl get secret ${SECRET} -n ${NS} -o jsonpath='{.data.tls\.key}' | base64 -d | openssl rsa -noout -modulus | md5)
echo "Match: $([ "$CERT_MOD" = "$KEY_MOD" ] && echo YES || echo NO)"

# Check if staging cert (not trusted in browsers)
kubectl get secret ${SECRET} -n ${NS} -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -issuer
# Staging issuer contains "(STAGING)" or "Fake LE"
```

| Problem | Cause | Diagnostic |
|---------|-------|------------|
| Browser shows "Not Trusted" | Using Let's Encrypt staging issuer | Check issuer URL in Issuer/ClusterIssuer spec; verify if it points to staging (`acme-staging-v02`) |
| Wrong SANs on cert | `dnsNames` in Certificate don't match Ingress hosts | Compare Certificate `dnsNames` with Ingress host rules; recommend updating Certificate to match |
| Cert issued but Ingress uses old one | Ingress controller cached old cert | Check Secret `tls.crt` issue date vs Ingress annotation; recommend restarting ingress controller or waiting for cache TTL |
| "certificate signed by unknown authority" | CA cert not in client trust store | Verify issuer chain in the certificate; recommend adding CA cert to client trust store or using `ca.crt` from Secret |

---

## MCP Tools Available

When the appropriate MCP servers are connected, prefer these over raw kubectl where available:

- `mcp__flux-operator-mcp__get_kubernetes_resources` - Query Certificates, CertificateRequests, Orders, Challenges, Issuers
- `mcp__flux-operator-mcp__get_kubernetes_logs` - Retrieve cert-manager controller, webhook, and cainjector logs
- `mcp__flux-operator-mcp__get_kubernetes_metrics` - Check cert-manager resource consumption

---

## Common Mistakes

| Mistake | Why It Fails | Instead |
|---------|--------------|---------|
| Testing with Let's Encrypt production | Hits rate limits quickly during debugging | Use `acme-staging-v02.api.letsencrypt.org` until everything works, then switch |
| Wildcard cert with HTTP-01 | HTTP-01 doesn't support wildcards, only specific domains | Use DNS-01 solver for wildcard certificates (`*.example.com`) |
| Forgetting port 80 for HTTP-01 | Let's Encrypt must reach `/.well-known/acme-challenge/` over HTTP | Ensure port 80 is open from the internet to the ingress controller |
| Using `cert-manager.io/issuer` for a ClusterIssuer | Ingress annotation looks for a namespace-scoped Issuer | Use `cert-manager.io/cluster-issuer` annotation instead |
| Deleting and recreating Certificate for renewal | Causes a new ACME order, consuming rate limits | Recommend using `kubectl cert-man

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [foxj77](https://github.com/foxj77)
- **Source:** [foxj77/claude-code-skills](https://github.com/foxj77/claude-code-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-foxj77-claude-code-skills-cert-manager-troubleshooting
- Seller: https://agentstack.voostack.com/s/foxj77
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.