TokenBrake

Runaway agent loops outrun billing alerts

Recursive or runaway LLM agent loops can generate excessive API and token costs long before traditional monitoring, quotas or billing alerts react. By the time a spend alert fires, the cost has already been incurred.

Hybrid local memory with distributed enforcement

• Express.js middleware that sits in front of LLM API calls
• Sliding Window Log evaluation using Redis sorted sets (ZSET)
• Redis-backed distributed protection across multiple instances
• Multi-tenant request isolation so one tenant cannot exhaust another
• Graceful local-memory fallback when Redis is unavailable

Containment as a product decision, not just a rate limiter

• Real-time anomaly containment on repeated, near-identical request patterns
• Operational alerting hooks so teams see containment events as they happen
• Configurable windows and thresholds per tenant and per route
• Explicit reliability trade-off: availability preserved via local fallback

Benchmarked, not marketed

Performance figures in the repository are benchmarks and design targets measured in a controlled environment. They are reported as benchmarks — not as production business outcomes or customer savings.

Benchmark / target — not a production metric.

Open-source middleware with a product point of view

TokenBrake demonstrates how an infrastructure problem becomes a product problem: who is protected, what fails open, what fails closed, and how an operator learns that containment happened.