Cloudhsm Diagnostics

AWS CloudHSM Diagnostics

When to use

Any AWS CloudHSM investigation where the console alone is insufficient — cluster and HSM management, client connectivity, cryptographic key operations, HSM user management, backup and restore, high availability, SDK integration, PKCS#11 library, JCE provider, or OpenSSL engine issues.

Investigation workflow

Step 1 — Collect and triage

aws cloudhsmv2 describe-clusters
aws cloudhsmv2 describe-clusters --filters clusterIds=
aws cloudhsmv2 list-tags --resource-id 

Step 2 — Domain deep dive

aws cloudhsmv2 describe-backups --filters clusterIds=
aws ec2 describe-security-groups --group-ids 
aws ec2 describe-subnets --subnet-ids 
# On client instance:
/opt/cloudhsm/bin/cloudhsm-cli cluster info
/opt/cloudhsm/bin/cloudhsm-cli user list

Step 3 — Detailed investigation

aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventSource,AttributeValue=cloudhsmv2.amazonaws.com --max-results 20
aws cloudwatch get-metric-statistics --namespace AWS/CloudHSM --metric-name HsmKeysSessionCount --dimensions Name=ClusterId,Value= --start-time  --end-time  --period 300 --statistics Average
# Check client logs:
# /opt/cloudhsm/run/cloudhsm-client.log

Read references/guardrails.md before concluding on any CloudHSM issue.

Tool quick reference

| Tool / API | When to use | |------------|-------------| | describe-clusters | Check cluster and HSM status | | describe-backups | Check backup status and history | | cloudhsm-cli | HSM-level user and key management | | Security Groups | Verify client-to-HSM connectivity | | CloudWatch Metrics | Monitor HSM health and usage | | CloudTrail | Audit management API calls | | Client logs | Debug connectivity and operation errors |

Gotchas: AWS CloudHSM

• CloudHSM is a single-tenant HSM. You manage HSM users, keys, and access. AWS manages hardware and firmware. AWS cannot access your keys or recover them if credentials are lost. Losing all crypto officer (CO) credentials means losing key management access permanently.
• Client connectivity requires: (1) CloudHSM client installed on EC2 instance, (2) instance in same VPC or peered VPC, (3) security group allows port 2223-2225 from client to HSM ENI, (4) client configuration points to cluster IP.
• HSM users are managed INSIDE the HSM, not via IAM. Three user types: PRECO (pre-crypto officer for initial setup), CO (crypto officer — manages users), CU (crypto user — performs crypto operations). IAM controls management plane (create/delete cluster), HSM users control data plane.
• Clusters should have 2+ HSMs in different AZs for HA. Keys are automatically synchronized across HSMs in a cluster. If one HSM fails, the client SDK automatically fails over to another HSM. Single-HSM clusters have no failover.
• Backups are encrypted and stored in S3 (managed by AWS). Backups contain all HSM users and keys. Restoring a backup creates a new cluster with the same users and keys. Backups are automatic (daily) and can be triggered manually.
• PKCS#11, JCE, and OpenSSL integrations use the CloudHSM client SDK. Each has specific library paths and configuration. Version mismatches between SDK components cause failures.

Anti-hallucination rules

1. Always cite specific cluster IDs, HSM IDs, or client log entries as evidence.
2. AWS CANNOT recover keys or reset HSM users. Never suggest contacting AWS for key recovery.
3. HSM users are NOT IAM users. Never confuse IAM and HSM user management.
4. Port 2223-2225 is required for client connectivity. Never use other ports.
5. Single-HSM clusters have NO failover. Never claim HA with one HSM.
6. Spend no more than 2 minutes on any single hypothesis. Pivot if inconclusive.

14 runbooks

| Category | IDs | Covers | |----------|-----|--------| | A — Cluster | A1–A2 | Cluster creation, HSM provisioning | | B — Connectivity | B1–B2 | Client connectivity, network issues | | C — Keys | C1–C2 | Key management, key operations | | D — Users | D1–D2 | User management, credential issues | | E — Backup | E1–E2 | Backup/restore, backup failures | | F — HA | F1–F2 | High availability, failover | | G — SDK | G1–G2 | PKCS#11, JCE provider | | H — Integration | H1–H2 | OpenSSL, SDK configuration | | Z — Catch-All | Z1 | General troubleshooting |

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: aws-samples
• Source: aws-samples/sample-ai-agent-skills
• License: MIT-0

Install and usage instructions live in the source repository linked above.