Skip to content

Operations Guide

This guide consolidates troubleshooting and operational procedures for zae-limiter deployments. Navigate using the interactive map below or jump directly to a topic.

flowchart LR OPS[Operations Guide] subgraph alerts[Alerts & Issues] LAMBDA[Lambda Aggregator] DYNAMO[DynamoDB] STREAMS[Stream Processing] VERSION[Version Errors] LIMITS[Rate Limits] end subgraph planned[Planned Operations] UPGRADE[Version Upgrade] LAMBDA_UPDATE[Lambda Update] SCALE_LIMITS[Adjust Limits] SCALE_DYNAMO[Scale DynamoDB] BACKUP[Backup & Restore] ROLLBACK[Emergency Rollback] end OPS --> alerts OPS --> planned click LAMBDA "lambda/" "Lambda troubleshooting" click DYNAMO "dynamodb/" "DynamoDB operations" click STREAMS "streams/" "Stream processing" click VERSION "version/" "Version management" click LIMITS "rate-limits/" "Rate limit issues" click UPGRADE "version/#upgrade-procedure" "Upgrade procedure" click LAMBDA_UPDATE "lambda/#lambda-redeployment" "Redeploy Lambda" click SCALE_LIMITS "rate-limits/#adjust-limits-at-runtime" "Adjust limits" click SCALE_DYNAMO "dynamodb/#scaling-procedures" "Scale capacity" click BACKUP "recovery/#dynamodb-backup-and-restore" "Backup & restore" click ROLLBACK "recovery/#emergency-rollback-decision-matrix" "Emergency rollback"

Alerts & Issues

Component Common Issues
Lambda Error rate, Duration/timeout, DLQ messages
DynamoDB Throttling, Capacity
Streams Iterator age, Processing lag
Version VersionMismatchError, IncompatibleSchemaError
Rate Limits Unexpected RateLimitExceeded, Limits not enforcing

Planned Operations

Operation Guides
Upgrades Version upgrade, Lambda update
Scaling Adjust rate limits, DynamoDB capacity
Recovery Emergency rollback, Backup/restore, PITR

Quick Reference

Symptom Go To
RateLimitExceeded unexpected Rate Limits
ProvisionedThroughputExceededException DynamoDB
DLQ messages accumulating Lambda
VersionMismatchError Version
High IteratorAge Streams
Need to rollback Recovery

CLI Diagnostic Commands

Command Description
zae-limiter status --name <name> Check stack status and resources
zae-limiter version --name <name> Show version information
zae-limiter check --name <name> Check client/infrastructure compatibility

CloudWatch Metrics Overview

Metric Namespace Threshold Guide
Errors AWS/Lambda > 1/5min Lambda
Duration AWS/Lambda > 80% timeout Lambda
IteratorAge AWS/Lambda > 30,000ms Streams
ReadThrottleEvents AWS/DynamoDB > 0 DynamoDB
WriteThrottleEvents AWS/DynamoDB > 0 DynamoDB
ApproximateNumberOfMessagesVisible AWS/SQS > 0 Lambda

Exception Reference

Exception Cause Guide
RateLimitExceeded Rate limit violated Rate Limits
RateLimiterUnavailable DynamoDB unavailable DynamoDB
EntityNotFoundError Entity doesn't exist Rate Limits
VersionMismatchError Client/Lambda version mismatch Version
IncompatibleSchemaError Major version difference Version
StackCreationError CloudFormation failed Recovery