Skip to content

DynamoDB Operations

This guide covers troubleshooting and operational procedures for DynamoDB capacity management and throttling issues.

Decision Tree

flowchart TD START([DynamoDB Issue]) --> Q1{What's the symptom?} Q1 -->|ReadThrottleEvents| A1[Check read patterns] Q1 -->|WriteThrottleEvents| A2[Check write patterns] Q1 -->|High latency| A3[Check capacity utilization] Q1 -->|Planning scale| A4[Capacity planning] A1 --> FIX1{Emergency?} A2 --> FIX1 FIX1 -->|Yes| EMERGENCY[Switch to on-demand] FIX1 -->|No| PLAN[Increase provisioned capacity] A3 --> CHECK[Check CloudWatch metrics] CHECK --> FIX1 A4 --> LINK([→ Performance Guide]) click A1 "#throttling" "Diagnose throttling" click A2 "#throttling" "Diagnose throttling" click A3 "#capacity-planning" "Check capacity" click EMERGENCY "#emergency-on-demand" "Switch to on-demand" click PLAN "#scaling-procedures" "Scale capacity" click CHECK "#diagnostic-queries" "View metrics" click LINK "../performance/" "Capacity planning guide"

Troubleshooting

Symptoms

  • ProvisionedThroughputExceededException errors
  • Increased latency on rate limit checks
  • CloudWatch throttle alarms triggered
  • RateLimiterUnavailable with FAIL_CLOSED mode

Diagnostic Steps

Check CloudWatch metrics:

# View throttle events (last hour)
aws cloudwatch get-metric-statistics \
  --namespace AWS/DynamoDB \
  --metric-name ReadThrottleEvents \
  --dimensions Name=TableName,Value=ZAEL-<name> \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --period 300 \
  --statistics Sum

Check capacity utilization:

aws cloudwatch get-metric-statistics \
  --namespace AWS/DynamoDB \
  --metric-name ConsumedReadCapacityUnits \
  --dimensions Name=TableName,Value=ZAEL-<name> \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --period 300 \
  --statistics Sum

Identify hot partitions (if Contributor Insights enabled):

aws dynamodb describe-contributor-insights \
  --table-name ZAEL-<name>

Throttling

Common Causes and Solutions

Cause Solution
Provisioned capacity too low Increase RCU/WCU or switch to on-demand
Hot partition Distribute entity IDs more evenly
Burst traffic Enable auto-scaling or use on-demand
GSI throttling Check GSI capacity separately

Read Throttling

Check which operations are throttling:

aws cloudwatch get-metric-statistics \
  --namespace AWS/DynamoDB \
  --metric-name ThrottledRequests \
  --dimensions Name=TableName,Value=ZAEL-<name> Name=Operation,Value=GetItem \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --period 300 \
  --statistics Sum

Write Throttling

Write throttling typically occurs during high-volume rate limiting or when the aggregator Lambda is processing many stream events.

Check write patterns:

aws cloudwatch get-metric-statistics \
  --namespace AWS/DynamoDB \
  --metric-name ConsumedWriteCapacityUnits \
  --dimensions Name=TableName,Value=ZAEL-<name> \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --period 60 \
  --statistics Sum

Capacity Planning

For detailed capacity calculations, see the Performance Tuning Guide.

Quick estimates:

Operation RCU WCU
acquire() ~2 ~4
acquire() with cascade ~4 ~8
available() ~2 ~0
Aggregator (per record) ~1 ~2

Procedures

Emergency Capacity Increase

Switch to on-demand capacity (immediate relief):

Billing Impact

On-demand pricing is typically 5-7x more expensive than provisioned capacity at steady state, but has no throttling.

aws dynamodb update-table \
  --table-name ZAEL-<name> \
  --billing-mode PAY_PER_REQUEST

Increase provisioned capacity:

aws dynamodb update-table \
  --table-name ZAEL-<name> \
  --provisioned-throughput ReadCapacityUnits=1000,WriteCapacityUnits=500

Scaling Procedures

Planned Capacity Scaling

Step 1: Analyze current usage

# Get average consumption over last 24 hours
aws cloudwatch get-metric-statistics \
  --namespace AWS/DynamoDB \
  --metric-name ConsumedReadCapacityUnits \
  --dimensions Name=TableName,Value=ZAEL-<name> \
  --start-time $(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --period 3600 \
  --statistics Average

Step 2: Calculate required capacity

Use the formulas from Performance Tuning:

  • RCU = (requests_per_second × 2) + (cascade_requests × 2)
  • WCU = (requests_per_second × 4) + (cascade_requests × 4)

Add 20% headroom for bursts.

Step 3: Apply changes

aws dynamodb update-table \
  --table-name ZAEL-<name> \
  --provisioned-throughput ReadCapacityUnits=<new_rcu>,WriteCapacityUnits=<new_wcu>

Step 4: Verify

aws dynamodb describe-table --table-name ZAEL-<name> \
  --query 'Table.ProvisionedThroughput'

Enable Auto-Scaling

Create scaling targets:

# Register read capacity target
aws application-autoscaling register-scalable-target \
  --service-namespace dynamodb \
  --resource-id "table/ZAEL-<name>" \
  --scalable-dimension "dynamodb:table:ReadCapacityUnits" \
  --min-capacity 5 \
  --max-capacity 1000

# Register write capacity target
aws application-autoscaling register-scalable-target \
  --service-namespace dynamodb \
  --resource-id "table/ZAEL-<name>" \
  --scalable-dimension "dynamodb:table:WriteCapacityUnits" \
  --min-capacity 5 \
  --max-capacity 500

Create scaling policies:

# Read capacity policy (target 70% utilization)
aws application-autoscaling put-scaling-policy \
  --service-namespace dynamodb \
  --resource-id "table/ZAEL-<name>" \
  --scalable-dimension "dynamodb:table:ReadCapacityUnits" \
  --policy-name "ZAEL-<name>-read-scaling" \
  --policy-type "TargetTrackingScaling" \
  --target-tracking-scaling-policy-configuration '{
    "TargetValue": 70.0,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "DynamoDBReadCapacityUtilization"
    }
  }'

Switch to On-Demand

When to use on-demand:

  • Unpredictable traffic patterns
  • New deployments without baseline data
  • Cost is less important than avoiding throttling

Switch from provisioned to on-demand:

aws dynamodb update-table \
  --table-name ZAEL-<name> \
  --billing-mode PAY_PER_REQUEST

Switch back to provisioned:

Cooldown Period

You can only switch billing modes once per 24 hours.

aws dynamodb update-table \
  --table-name ZAEL-<name> \
  --billing-mode PROVISIONED \
  --provisioned-throughput ReadCapacityUnits=100,WriteCapacityUnits=50

Verification

After capacity changes, monitor for 15-30 minutes:

# Watch throttle events
watch -n 30 "aws cloudwatch get-metric-statistics \
  --namespace AWS/DynamoDB \
  --metric-name ReadThrottleEvents \
  --dimensions Name=TableName,Value=ZAEL-<name> \
  --start-time \$(date -u -d '30 minutes ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time \$(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --period 60 \
  --statistics Sum"