api模型检测
Вы не можете выбрать более 25 тем Темы должны начинаться с буквы или цифры, могут содержать дефисы(-) и должны содержать не более 35 символов.

3.2 KiB

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Build Commands

cargo build                          # debug build
cargo build --release                # release build
cargo test                           # run all tests
cargo test <test_name>               # run a single test
cargo clippy --all-targets -- -D warnings  # lint
cargo fmt --check                    # format check
cargo fmt                            # auto-format

Tracing is controlled via RUST_LOG env var (uses tracing-subscriber with env-filter).

Architecture

This is a Rust CLI (clap derive) for testing LLM relay compatibility, benchmark accuracy, and RPM rate-limit behavior. It supports OpenAI-compatible and Anthropic-compatible relay protocols.

Module Layout

  • cli.rs — CLI definition and command dispatch. Contains all subcommand handlers (check, dataset fetch, bench, rpm). The RPM command builds a schedule of timed probes, executes them concurrently, then aggregates results into a mode-specific report.
  • config.rs — YAML config loading with ${ENV_VAR} expansion. Provider tokens are resolved lazily (only the requested provider’s env vars need to be set). AppConfig::load() is the entry point.
  • runner.rs — Thin orchestrator that dispatches to the correct protocol adapter and measures elapsed time. run_model_request() is the single call site for all LLM requests.
  • protocols/ — Protocol adapters (openai.rs, anthropic.rs). Each implements send(client, request) -> ModelResponse. Shared URL normalization and error extraction live in protocols/mod.rs.
  • benchmarks/ — Dataset fetching from Hugging Face (fetch_dataset), case loaders (aime.rs, gpqa.rs), and answer judging (judge.rs). AIME rows are normalized from HF API JSON to local JSONL.
  • metrics.rs — HDR histogram-based latency tracking plus success/failure/accuracy counters. Metrics::summary() produces the final MetricsSummary.
  • rpm_modes.rs — Schedule generators for each RPM test mode (sustained, burst, token-bucket, sliding-window, window-boundary, diagnose). Each returns a Vec<ScheduledProbe> with offsets and phase labels.
  • report.rs — Report structs and JSON serialization. Reports are written to reports/ directory.

Request Flow

Cli::parse()cli::dispatch() → builds ModelRequestrunner::run_model_request()protocols::{openai,anthropic}::send() → returns ModelResponse.

For RPM tests, requests are scheduled via tokio::time::sleep_until with stream::buffer_unordered for concurrency control.

Configuration

Config is loaded from config.yaml (YAML). Values like ${ENV_NAME} are expanded at load time. Provider tokens are expanded only when that provider is resolved, so unused providers don’t require their env vars to be set.

Conventions

  • Rust edition 2024
  • Error handling: anyhow::Result for application errors, thiserror for typed errors in config.rs
  • Async runtime: tokio multi-thread
  • HTTP client: reqwest with rustls-tls (no OpenSSL dependency)
  • API tokens are redacted in Debug impls
  • Tests use tempfile for filesystem isolation and wiremock for HTTP mocking