# CLAUDE.md This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Build Commands ```bash cargo build # debug build cargo build --release # release build cargo test # run all tests cargo test # run a single test cargo clippy --all-targets -- -D warnings # lint cargo fmt --check # format check cargo fmt # auto-format ``` Tracing is controlled via `RUST_LOG` env var (uses tracing-subscriber with env-filter). ## Architecture This is a Rust CLI (`clap` derive) for testing LLM relay compatibility, benchmark accuracy, and RPM rate-limit behavior. It supports OpenAI-compatible and Anthropic-compatible relay protocols. ### Module Layout - `cli.rs` — CLI definition and command dispatch. Contains all subcommand handlers (`check`, `dataset fetch`, `bench`, `rpm`). The RPM command builds a schedule of timed probes, executes them concurrently, then aggregates results into a mode-specific report. - `config.rs` — YAML config loading with `${ENV_VAR}` expansion. Provider tokens are resolved lazily (only the requested provider's env vars need to be set). `AppConfig::load()` is the entry point. - `runner.rs` — Thin orchestrator that dispatches to the correct protocol adapter and measures elapsed time. `run_model_request()` is the single call site for all LLM requests. - `protocols/` — Protocol adapters (`openai.rs`, `anthropic.rs`). Each implements `send(client, request) -> ModelResponse`. Shared URL normalization and error extraction live in `protocols/mod.rs`. - `benchmarks/` — Dataset fetching from Hugging Face (`fetch_dataset`), case loaders (`aime.rs`, `gpqa.rs`), and answer judging (`judge.rs`). AIME rows are normalized from HF API JSON to local JSONL. - `metrics.rs` — HDR histogram-based latency tracking plus success/failure/accuracy counters. `Metrics::summary()` produces the final `MetricsSummary`. - `rpm_modes.rs` — Schedule generators for each RPM test mode (sustained, burst, token-bucket, sliding-window, window-boundary, diagnose). Each returns a `Vec` with offsets and phase labels. - `report.rs` — Report structs and JSON serialization. Reports are written to `reports/` directory. ### Request Flow `Cli::parse()` → `cli::dispatch()` → builds `ModelRequest` → `runner::run_model_request()` → `protocols::{openai,anthropic}::send()` → returns `ModelResponse`. For RPM tests, requests are scheduled via `tokio::time::sleep_until` with `stream::buffer_unordered` for concurrency control. ### Configuration Config is loaded from `config.yaml` (YAML). Values like `${ENV_NAME}` are expanded at load time. Provider tokens are expanded only when that provider is resolved, so unused providers don't require their env vars to be set. ## Conventions - Rust edition 2024 - Error handling: `anyhow::Result` for application errors, `thiserror` for typed errors in `config.rs` - Async runtime: tokio multi-thread - HTTP client: reqwest with rustls-tls (no OpenSSL dependency) - API tokens are redacted in Debug impls - Tests use `tempfile` for filesystem isolation and `wiremock` for HTTP mocking