Hierarchical Limits¶

zae-limiter supports two-level hierarchies for rate limiting, enabling patterns like:

Project → API Keys: Limit total project usage while also limiting individual keys
Organization → Users: Organization-wide limits with per-user quotas
Tenant → Services: Multi-tenant limits with service-level controls

Creating a Hierarchy¶

# Create parent entity (project)
await limiter.create_entity(
    entity_id="project-1",
    name="Production Project",
)

# Create child entities (API keys) with cascade enabled
await limiter.create_entity(
    entity_id="key-abc",
    parent_id="project-1",
    name="Web Application Key",
    cascade=True,  # Enforce parent limits on every acquire
)

await limiter.create_entity(
    entity_id="key-xyz",
    parent_id="project-1",
    name="Mobile App Key",
    cascade=True,
)

Same-Namespace Constraint

Parent and child entities must belong to the same namespace. Cross-namespace hierarchies are not supported. If you use multiple namespaces, create separate entity hierarchies within each namespace.

Cascade Mode¶

Create entities with cascade=True to apply rate limits to both the child and parent on every acquire() call:

# Cascade is set once at entity creation
await limiter.create_entity(
    entity_id="key-abc",
    parent_id="project-1",
    cascade=True,  # All acquire() calls will also check parent
)

# acquire() automatically cascades to parent — no flag needed
async with limiter.acquire(
    entity_id="key-abc",
    resource="gpt-4",
    limits=[
        Limit.per_minute("tpm", 10_000),  # Per-key limit
    ],
    consume={"tpm": 500},
) as lease:
    await call_api()

Performance Impact

Cascade mode adds overhead: +1 GetEntity + parent bucket operations. Only enable when hierarchical enforcement is needed. See Batch Operation Patterns for optimization strategies.

Speculative Writes with Cascade

With speculative_writes=True, the first cascade acquire completes in 2 sequential round trips (0 RCU, 2 WCU). After the first acquire populates the entity metadata cache, subsequent cascade acquires issue child and parent writes in parallel -- completing in just 1 round trip (0 RCU, 2 WCU) with the same cost but lower latency. If the parent needs refill, a deferred compensation optimization avoids unnecessary child compensation by attempting a parent-only slow path first. See Speculative Writes for details.

What happens:

Check if key-abc has capacity (10k tpm)
Check if project-1 has capacity (uses same limits)
If both pass, consume from both atomically
If either fails, reject with details about which limit was exceeded

Different Limits Per Level¶

Set different limits for parents and children:

# Set project-level limits (higher)
await limiter.set_limits(
    entity_id="project-1",
    limits=[
        Limit.per_minute("tpm", 100_000),  # 100k for entire project
    ],
)

# Set key-level limits (lower)
await limiter.set_limits(
    entity_id="key-abc",
    limits=[
        Limit.per_minute("tpm", 10_000),  # 10k per key
    ],
)

# acquire() auto-cascades because key-abc was created with cascade=True
async with limiter.acquire(
    entity_id="key-abc",
    resource="gpt-4",
    limits=None,  # Auto-resolves from stored config
    consume={"tpm": 500},
) as lease:
    await call_api()

Understanding Cascade Behavior¶

Without Cascade¶

# Entity created without cascade (default)
await limiter.create_entity(entity_id="key-abc", parent_id="project-1")

# Only checks/consumes from key-abc
async with limiter.acquire(
    entity_id="key-abc",
    resource="gpt-4",
    limits=[Limit.per_minute("tpm", 10_000)],
    consume={"tpm": 500},
) as lease:
    ...

With Cascade¶

# Entity created with cascade enabled
await limiter.create_entity(entity_id="key-abc", parent_id="project-1", cascade=True)

# Checks/consumes from BOTH key-abc AND project-1
async with limiter.acquire(
    entity_id="key-abc",
    resource="gpt-4",
    limits=[Limit.per_minute("tpm", 10_000)],
    consume={"tpm": 500},
) as lease:
    ...

Cascade per Resource¶

cascade=True on create_entity() applies to every resource the entity uses. Sometimes one entity needs different answers on different resources. Take an LLM gateway:

Resource	Cascade	Why
One per model (`gpt-4`, `claude-sonnet`, …)	on	The org's per-model limit protects the provider's quota
`llm` (a shared cost budget)	off	The budget is per user; debiting the org on every call costs a write for nothing

A cascade policy on resource or entity config decides this per resource. It is on, off, or unset (inherit), and resolves like disabled:

Entity config for this resource
Entity config for all resources (_default_)
Resource config

The first level that sets a value wins. When no level sets one, the entity's own create_entity(cascade=...) flag applies, so nothing changes until you set a policy. An entity with no parent never cascades, whatever the policy says. There is no system-level policy.

The policy methods live on Repository, like the disable methods:

await limiter.create_entity(entity_id="org-acme")
await limiter.create_entity(entity_id="user-alice", parent_id="org-acme", cascade=True)

# The org's own limit for the model: this is what a cascade debits
await limiter.set_limits("org-acme", [Limit.per_minute("tpm", 100_000)], resource="gpt-4")

# Model limits count against the org; the shared budget stays per user
await repo.set_resource_cascade("gpt-4", True)
await repo.set_resource_cascade("llm", False)

# One entity, one resource: an explicit override
await repo.set_entity_cascade("user-alice", True, resource="llm")

# Back to inheriting
await repo.clear_entity_cascade("user-alice", resource="llm")

set_resource_defaults() and set_limits() also take a cascade= keyword, and get_resource_cascade() / get_entity_cascade() read a level's own value (None when it inherits).

The same from the CLI (see Cascade Policy per Resource):

zae-limiter resource set-cascade gpt-4 on
zae-limiter resource set-cascade llm off
zae-limiter entity set-cascade user-alice on --resource llm
zae-limiter entity clear-cascade user-alice --resource llm

Or in a limits manifest, applied with limits apply:

resources:
  gpt-4:
    cascade: true
    limits:
      tpm: {capacity: 10000}
  llm:
    cascade: false
    limits:
      cost: {capacity: 500}

entities:
  org-acme:
    resources:
      gpt-4:
        limits:
          tpm: {capacity: 100000}

Give the parent its own limits

A cascading acquire debits the parent against the parent's own limits for that resource. If the parent has none, it falls back to the resource defaults — the numbers written for one user. limits plan warns when a manifest turns cascade on for a resource and no entity in it has its own limits there.

Cost: no extra reads per call. The fast path reads the policy from the bucket item it already writes, and the slow path reads it from the config items it already fetches. Changing a policy restamps the existing buckets it reaches (one write each), like disabling a resource.

Version requirement: setting or clearing a policy needs a stack whose Lambdas are 0.16.0 or later, and raises the stack's minimum client version to 0.16.0. Run zae-limiter upgrade first if the call is refused.

See ADR-146 for the full design.

Error Handling with Hierarchies¶

When an entity has cascade enabled, RateLimitExceeded includes statuses for all entities:

try:
    async with limiter.acquire(
        entity_id="key-abc",  # Has cascade=True from create_entity()
        resource="gpt-4",
        limits=[Limit.per_minute("rpm", 100)],
        consume={"rpm": 1},
    ):
        pass
except RateLimitExceeded as e:
    for status in e.statuses:
        print(f"Entity: {status.entity_id}")
        print(f"  Limit: {status.limit_name}")
        print(f"  Available: {status.available}")
        print(f"  Exceeded: {status.exceeded}")

Use Cases¶

Multi-Tenant SaaS¶

# Tenant sustains 1M tokens/day, dripping continuously.
# For an allowance that returns in one lump at midnight, use
# Limit.quota("tpd", 1_000_000, cron="0 0 * * *", tz=...) instead.
await limiter.set_limits(
    entity_id="tenant-acme",
    limits=[Limit.per_day("tpd", 1_000_000)],
)

# Create user under tenant with cascade enabled
await limiter.create_entity(entity_id="user-123", parent_id="tenant-acme", cascade=True)

# Each user sustains 100k tokens/day
await limiter.set_limits(
    entity_id="user-123",
    limits=[Limit.per_day("tpd", 100_000)],
)

# Rate limit user — auto-cascades to tenant
# limits=None auto-resolves from stored config
async with limiter.acquire(
    entity_id="user-123",
    resource="gpt-4",
    limits=None,
    consume={"tpm": 500},
) as lease:
    ...

API Key Management¶

# Project limit: 10k RPM
await limiter.set_limits(
    entity_id="project-prod",
    limits=[Limit.per_minute("rpm", 10_000)],
)

# Production key: 5k RPM (half of project)
await limiter.set_limits(
    entity_id="key-prod",
    limits=[Limit.per_minute("rpm", 5_000)],
)

# Staging key: 1k RPM
await limiter.set_limits(
    entity_id="key-staging",
    limits=[Limit.per_minute("rpm", 1_000)],
)

Moving an Entity to a New Parent¶

Use set_parent() to move an entity under another parent, for example a user who changes organisation, or pass None to make it a root entity:

await limiter.create_entity(entity_id="org-north")
await limiter.create_entity(entity_id="org-south")
await limiter.create_entity(entity_id="user-bob", parent_id="org-north", cascade=True)

# Bob changes organisation: his calls now count against org-south
await limiter.set_parent("user-bob", "org-south")

# Bob no longer has a parent
await limiter.set_parent("user-bob", None)

The move takes effect on the next acquire in every process:

Its buckets are restamped at once. set_parent() returns how many.
Other processes learn it from the next call. A process that still has the old parent cached debits it once, sees the new parent on the entity's bucket, refunds the old parent and debits the new one in the same call.
A read of the entity from before the move is caught. When a call's read of the entity is older than its bucket's stamp, the call reads the entity again, strongly consistent, and debits the new parent.
A bucket created while the move lands cannot keep the old parent. Creating a bucket is one transaction that also checks the entity's record still names the parent it read; if a move landed in between, the acquire re-reads the entity and retries once. This needs dynamodb:ConditionCheckItem on the table for every principal that acquires: the policies the CloudFormation template creates grant it, and custom IAM must add it.
Nothing already debited moves. Usage charged to the old parent stays there, including usage from leases opened before the move: their adjustments and rollbacks land on the old parent, where the tokens were taken.
Cascade follows the policy, not the move. The new parent is debited wherever the entity cascades — its cascade policy, or the cascade flag it was created with. A root entity created with cascade=False and moved under a parent does not cascade until you set a policy, for example set_entity_cascade("user-123", True).

set_parent() raises EntityNotFoundError when the entity or the new parent does not exist, and ValidationError when the move would make the entity its own ancestor. If restamping stops part-way it raises FanoutIncomplete; the move itself landed, and calling set_parent() again with the same arguments finishes it. It needs a stack whose Lambdas are 0.17.0 or later (VersionMismatchError otherwise; run zae-limiter upgrade). Restart every client after upgrading, before the first move: a process that opened the stack with an older version keeps debiting the old parent until it reopens it. Each move is recorded as an entity_parent_changed audit event.

Limitations¶

Two levels only: Parent → Child (no grandparents)
Single parent: Each entity can have at most one parent at a time; set_parent() moves it
Cascade defaults per entity: Set cascade=True on create_entity() to enable it for every resource; a cascade policy can override it per resource

Next Steps¶

Per-Resource Limits and a Shared Budget - Users under organisations, cascade per resource and a per-user budget, end to end
LLM Integration - Token estimation patterns
Unavailability Handling - Handling service outages
API Reference - Complete API documentation