new-api-check/docs/superpowers/plans/2026-05-06-rpm-limiter-modes-implementation.md

RPM Limiter Modes Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Extend rpm testing from one sustained schedule into six explicit limiter test modes: sustained, burst, token-bucket, sliding-window, window-boundary, and diagnose.

Architecture: Keep mode-specific scheduling and inference in a focused module, while cli remains responsible for argument parsing and command dispatch. Reports should preserve the current RPM summary shape and add mode-specific detail without storing raw prompts or secrets.

Tech Stack: Rust 2024, existing clap, tokio, futures, serde, chrono, hdrhistogram, anyhow.

---

File Structure

• Modify src/cli.rs: add --mode, mode-specific flags, dispatch to mode runner, and keep current sustained behavior as default.
• Create src/rpm_modes.rs: mode enum, schedules, probe plans, limiter inference, and unit tests.
• Modify src/report.rs: extend RPM reports with mode, optional burst/probe/window/diagnose details, actual RPM, and inferred limiter.
• Modify src/main.rs: register rpm_modes module.
• Modify README.md: document RPM modes and example commands.

Mode Semantics

• sustained: current default. Starts one request every 60 / rpm seconds for duration. Requests may overlap.
• burst: starts burst requests at t=0 and reports immediate success/failure/latency/error behavior.
• token-bucket: starts burst requests at t=0, then probes refill behavior for probe_seconds. Probe rate defaults to the expected refill rate from rpm, rounded up to at least one probe per second.
• sliding-window: starts burst requests at t=0, then sends low-rate probes for probe_seconds to observe whether recovery happens near the rolling 60 second boundary.
• window-boundary: waits until the next minute boundary test point, sends one batch before the boundary and one batch after it. This tests fixed-window reset behavior.
• diagnose: runs a bounded combined probe using burst, refill probes, and boundary signals, then writes a best-effort inference: token_bucket, fixed_window, sliding_window, or unknown.

CLI Shape

Keep the old command valid:

cargo run -- rpm --provider anthropic --rpm 120 --duration 60s --prompt "hello"

Equivalent explicit command:

cargo run -- rpm --mode sustained --provider anthropic --rpm 120 --duration 60s --prompt "hello"

New commands:

cargo run -- rpm --mode burst --provider anthropic --burst 120 --prompt "hello"
cargo run -- rpm --mode token-bucket --provider anthropic --rpm 120 --burst 120 --probe-seconds 30 --prompt "hello"
cargo run -- rpm --mode sliding-window --provider anthropic --rpm 120 --burst 120 --probe-seconds 90 --prompt "hello"
cargo run -- rpm --mode window-boundary --provider anthropic --rpm 120 --burst 120 --prompt "hello"
cargo run -- rpm --mode diagnose --provider anthropic --rpm 120 --burst 120 --probe-seconds 90 --prompt "hello"

Argument defaults:

• --mode sustained
• --duration 60s, required only by sustained
• --burst <rpm> default for burst-style modes when omitted
• --probe-seconds 90 for sliding-window and diagnose
• --probe-seconds 30 for token-bucket
• --window-offset-ms 500 for window-boundary, meaning send before/after batches around the next minute boundary
• --concurrency should cap in-flight requests for all modes; default can be burst or a safe high value, but must never be zero

Report Shape

Extend RpmReport with:

{
  "mode": "token-bucket",
  "provider": "anthropic",
  "model": "claude-test",
  "run": {
    "started_at": "2026-05-06T15:30:12Z",
    "duration_ms": 60000,
    "target_rpm": 120,
    "actual_rpm": 118.5,
    "temperature": 0.0,
    "max_tokens": 1024
  },
  "summary": {
    "actual_requests": 120,
    "success": 118,
    "failure": 2,
    "latency_ms": {
      "p50": 800,
      "p95": 1500,
      "p99": 2200
    }
  },
  "mode_detail": {
    "burst": {
      "sent": 120,
      "success": 118,
      "failure": 2
    },
    "refill_probe": [
      {
        "second": 1,
        "sent": 2,
        "success": 2,
        "failure": 0
      }
    ],
    "inference": {
      "likely_limiter": "token_bucket",
      "confidence": "medium",
      "signals": [
        "burst accepted most initial requests",
        "probe success approximated 2.00 req/s refill"
      ]
    }
  },
  "errors": []
}

Rules:

• Do not serialize raw prompt.
• Do not serialize tokens.
• Include mode at top level.
• Include actual_rpm for every mode where duration is meaningful.
• Include mode detail only when relevant.
• Existing sustained reports remain easy to read.

Task 1: Mode Types And Scheduling

Files:

• Create: src/rpm_modes.rs

• Modify: src/main.rs

• Step 1: Add mode enum and parser

Implement RpmMode with clap::ValueEnum:

#[derive(Debug, Clone, Copy, PartialEq, Eq, clap::ValueEnum)]
#[value(rename_all = "kebab-case")]
pub enum RpmMode {
    Sustained,
    Burst,
    TokenBucket,
    SlidingWindow,
    WindowBoundary,
    Diagnose,
}

• Step 2: Add scheduling helpers

Implement:

pub fn sustained_schedule(duration: Duration, rpm: u32) -> Vec<Duration>;
pub fn burst_schedule(burst: u32) -> Vec<Duration>;
pub fn token_bucket_schedule(rpm: u32, burst: u32, probe_seconds: u64) -> Vec<ScheduledProbe>;
pub fn sliding_window_schedule(burst: u32, probe_seconds: u64) -> Vec<ScheduledProbe>;
pub fn window_boundary_plan(now: DateTime<Utc>, burst: u32, offset_ms: u64) -> WindowBoundaryPlan;

Use explicit structs for ScheduledProbe and WindowBoundaryPlan; include phase labels such as burst, refill_probe, sliding_probe, before_boundary, and after_boundary.

• Step 3: Add schedule tests

Test:

• 120 RPM sustained creates starts at 0ms, 500ms, 1000ms for a short duration.

• Burst 5 creates five 0ms starts.

• Token bucket 120 RPM with 120 burst and 2 probe seconds has 120 burst starts plus about 4 refill probes.

• Sliding window probe lasts through the requested probe seconds.

• Window boundary plan places before and after batches around a minute boundary.

• Step 4: Register module

Add mod rpm_modes; to src/main.rs.

• Step 5: Verify and commit

Run:

cargo fmt
cargo test rpm_modes

Commit:

git add src/rpm_modes.rs src/main.rs
git commit -m "feat: add rpm mode schedules"

Task 2: CLI Wiring And Mode Execution

Files:

• Modify: src/cli.rs

• Modify: src/rpm_modes.rs

• Step 1: Extend CLI args

Add to Command::Rpm:

#[arg(long, value_enum, default_value_t = RpmMode::Sustained)]
mode: RpmMode,
#[arg(long)]
burst: Option<u32>,
#[arg(long)]
probe_seconds: Option<u64>,
#[arg(long, default_value_t = 500)]
window_offset_ms: u64,
#[arg(long)]
concurrency: Option<usize>,

Keep existing rpm, duration, and prompt arguments compatible. duration may remain a string with default 60s; non-sustained modes can ignore it unless needed.

• Step 2: Build a request runner helper

Refactor RPM execution so all modes share:

async fn run_scheduled_requests(
    protocol: ProtocolKind,
    request: ModelRequest,
    starts: Vec<ScheduledRequest>,
    max_in_flight: usize,
) -> Vec<ScheduledResult>;

Each result should retain phase/second metadata for mode-detail reporting.

• Step 3: Implement mode execution

Mode behavior:

• sustained: use current behavior with sustained_schedule.

• burst: use burst_schedule.

• token-bucket: burst at 0, then refill probes based on rpm.

• sliding-window: burst at 0, then one probe per second until probe_seconds.

• window-boundary: sleep until planned before-boundary start, send before batch, then after batch.

• diagnose: run bounded burst/refill/boundary probes and produce inference.

• Step 4: Add CLI behavior tests

Add tests for:

• Old RPM command defaults to sustained.

• --mode token-bucket parses.

• --mode window-boundary --window-offset-ms 250 parses.

• Invalid zero --burst and zero --rpm are rejected by execution validation.

• Step 5: Verify and commit

Run:

cargo fmt
cargo test cli::tests rpm_modes
cargo test

Commit:

git add src/cli.rs src/rpm_modes.rs
git commit -m "feat: run rpm limiter modes"

Task 3: Report Extensions And Inference

Files:

• Modify: src/report.rs

• Modify: src/cli.rs

• Modify: src/rpm_modes.rs

• Step 1: Extend report structs

Add:

pub enum LimiterInferenceKind {
    TokenBucket,
    FixedWindow,
    SlidingWindow,
    Unknown,
}

pub struct LimiterInferenceReport {
    pub likely_limiter: LimiterInferenceKind,
    pub confidence: String,
    pub signals: Vec<String>,
}

pub struct RpmModeDetailReport {
    pub burst: Option<PhaseSummaryReport>,
    pub refill_probe: Vec<ProbeSecondReport>,
    pub sliding_probe: Vec<ProbeSecondReport>,
    pub window_boundary: Option<WindowBoundaryReport>,
    pub inference: Option<LimiterInferenceReport>,
}

Use serde rename attributes so JSON uses snake_case values like token_bucket.

• Step 2: Summarize phases

Add helpers to group scheduled results by phase and second:

• burst sent/success/failure

• refill probe per second

• sliding probe per second

• boundary before/after sent/success/failure

• Step 3: Implement diagnose inference

Best-effort rules:

• If boundary after batch succeeds much more than before batch, infer fixed_window with medium confidence.
• If refill probes recover near rpm / 60 requests per second after burst, infer token_bucket with medium confidence.
• If probes mostly fail until near 60 seconds after burst, infer sliding_window with medium confidence.
• Otherwise infer unknown with low confidence.

Do not overclaim; include signals in report.

• Step 4: Add report tests

Test:

• RPM report serializes mode.

• RPM report does not serialize raw prompt.

• Token bucket report includes burst and refill probe fields.

• Diagnose inference serializes expected snake_case limiter value.

• Step 5: Verify and commit

Run:

cargo fmt
cargo test report rpm_modes
cargo test

Commit:

git add src/report.rs src/cli.rs src/rpm_modes.rs
git commit -m "feat: report rpm limiter mode details"

Task 4: Documentation And Final Verification

Files:

• Modify: README.md

• Step 1: Document six modes

Add a section for:

• sustained: stable RPM.

• burst: instant burst capacity.

• token-bucket: burst plus refill probe.

• sliding-window: rolling 60 second recovery probe.

• window-boundary: fixed minute boundary probe.

• diagnose: combined unknown-mode diagnosis.

• Step 2: Document interpretation caveats

Mention that real LLM backends often combine RPM, TPM, concurrency, account-level, model-level, and region-level limits. Reports are strong signals, not perfect proof.

• Step 3: Final verification

Run:

cargo fmt --check
cargo test
cargo clippy --all-targets -- -D warnings
cargo run -- rpm --help

Do not run live network tests unless the user explicitly asks.

• Step 4: Commit

git add README.md
git commit -m "docs: document rpm limiter modes"

Self-Review

• Spec coverage: covers all six requested modes and keeps existing sustained behavior compatible.
• Document scan: no unresolved planning gaps.
• Security: reports must not store prompts or tokens.
• Testability: schedule, parser, report, and inference tests avoid network calls.