Bedrock Diagnostics

Amazon Bedrock Diagnostics

When to use

Any Amazon Bedrock investigation where the console alone is insufficient — model access and invocation issues, knowledge base configuration, agent orchestration, custom model training, guardrails configuration, performance optimization, security setup, or service integration.

Investigation workflow

Step 1 — Collect and triage

aws bedrock list-foundation-models --query 'modelSummaries[*].{Id:modelId,Name:modelName,Provider:providerName,Status:modelLifecycle.status}'
version: "1.0.0"
last_updated: "2025-04-12"
aws bedrock get-model-invocation-logging-configuration
aws bedrock list-custom-models
aws bedrock-agent list-agents --query 'agentSummaries[*].{Id:agentId,Name:agentName,Status:agentStatus}'
version: "1.0.0"
last_updated: "2025-04-12"

Step 2 — Domain deep dive

aws bedrock-agent list-knowledge-bases --query 'knowledgeBaseSummaries[*].{Id:knowledgeBaseId,Name:name,Status:status}'
version: "1.0.0"
last_updated: "2025-04-12"
aws bedrock-agent get-knowledge-base --knowledge-base-id 
aws bedrock get-guardrail --guardrail-identifier 
aws bedrock list-provisioned-model-throughputs

Step 3 — Detailed investigation

aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventSource,AttributeValue=bedrock.amazonaws.com --max-results 20
aws cloudwatch get-metric-statistics --namespace AWS/Bedrock --metric-name Invocations --dimensions Name=ModelId,Value= --start-time  --end-time  --period 300 --statistics Sum
aws bedrock-agent get-agent --agent-id 
aws bedrock-agent list-agent-action-groups --agent-id  --agent-version DRAFT

Read references/bedrock-guardrails.md before concluding on any Bedrock issue.

Tool quick reference

| Tool / API | When to use | |------------|-------------| | bedrock list-foundation-models | Check available models and status | | bedrock invoke-model | Test model invocation directly | | bedrock-agent list-knowledge-bases | List knowledge bases and status | | bedrock-agent get-knowledge-base | Get KB configuration details | | bedrock-agent list-agents | List agents and status | | bedrock-agent get-agent | Get agent configuration | | bedrock get-guardrail | Check guardrail configuration | | bedrock list-provisioned-model-throughputs | Check provisioned throughput | | bedrock list-custom-models | List fine-tuned models | | CloudWatch metrics | Monitor invocation counts, latency, errors |

Gotchas: Amazon Bedrock

• Model access must be explicitly requested and approved. Each foundation model requires separate access approval in the Bedrock console. Access is per-region. Without approval, InvokeModel returns AccessDeniedException.
• On-demand throughput has rate limits per model. Each model has different tokens-per-minute (TPM) and requests-per-minute (RPM) limits. Exceeding limits returns ThrottlingException. Provisioned throughput provides dedicated capacity.
• Knowledge base sync is asynchronous. After creating a KB or updating data sources, you must trigger a sync. Sync can take minutes to hours depending on data volume. Queries return stale results until sync completes.
• Agents use a multi-step orchestration loop. Agents break down tasks into steps, invoke action groups, and synthesize responses. Orchestration can loop if the agent cannot determine the next step. Max iterations and timeout settings control this.
• Guardrails evaluate both input and output. Content filters, topic denials, and word filters apply to both user input and model output. Overly restrictive guardrails block legitimate use cases. Test guardrails thoroughly before production.
• Custom model training requires specific S3 data format. Training data must be in JSONL format with specific schema. Training can take hours. Failed training jobs often stem from data format issues or insufficient IAM permissions.
• Provisioned throughput is billed even when idle. Provisioned model throughput provides dedicated capacity but incurs charges regardless of usage. Commitment terms (1-month, 6-month) affect pricing. No auto-scaling.
• VPC endpoints are required for private access. Bedrock API calls from private subnets require VPC endpoints. Both bedrock and bedrock-runtime endpoints may be needed. Security groups must allow HTTPS traffic.
• Streaming responses require specific client handling. InvokeModelWithResponseStream returns chunks. Client must handle the event stream protocol. Timeouts must account for full response generation time.

Anti-hallucination rules

1. Always cite specific model IDs, KB IDs, agent IDs, or API responses as evidence.
2. Model access requires explicit approval. Never assume a model is accessible.
3. On-demand has rate limits. Never claim unlimited throughput without provisioned capacity.
4. KB sync is async. Never assume data is immediately available after upload.
5. Guardrails evaluate both input AND output. Never claim they only filter one direction.
6. Spend no more than 2 minutes on any single hypothesis. Pivot if inconclusive.

24 runbooks

| Category | IDs | Covers | |----------|-----|--------| | A — Model | A1-A3 | Model access, invocation errors, throttling | | B — Knowledge Base | B1-B3 | KB creation, sync failures, retrieval issues | | C — Agents | C1-C3 | Agent creation, action group errors, orchestration | | D — Fine-Tuning | D1-D2 | Custom model training, provisioned throughput | | E — Guardrails | E1-E2 | Content filtering, topic denial | | F — Performance | F1-F2 | Latency, token limits | | G — Security | G1-G2 | IAM permissions, VPC config | | H — Integration | H1-H2 | Lambda integration, streaming responses | | Z — Catch-All | Z1 | General troubleshooting |

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: aws-samples
• Source: aws-samples/sample-ai-agent-skills
• License: MIT-0

Install and usage instructions live in the source repository linked above.