liushen
ffb25b0126
Merge pull request 'codex/lq-token-test-init' ( #2 ) from codex/lq-token-test-init into main
Reviewed-on: https://git.malls.iformall.com/server/new-api-check/pulls/2
4 дней назад
orangels
c346233072
feat: record stream chunk timings in reports
4 дней назад
orangels
1f5bdb26f6
feat: group reports by local date
5 дней назад
orangels
506434da6a
fix: use local time for raw debug logs
5 дней назад
orangels
4e0d85205a
feat: group raw debug logs by date
5 дней назад
orangels
70ced01112
feat: record response debug cases in reports
5 дней назад
orangels
a15a62264a
fix: set bench max tokens default to 64k
5 дней назад
orangels
1243558bc8
fix: include partial stream in interrupted debug logs
5 дней назад
orangels
739524628c
feat: report effective benchmark request params
5 дней назад
liushen
91e863b448
Merge pull request 'codex/lq-token-test-init' ( #1 ) from codex/lq-token-test-init into main
Reviewed-on: https://git.malls.iformall.com/server/new-api-check/pulls/1
5 дней назад
orangels
986dc4e088
fix: restore bench max tokens default
5 дней назад
orangels
b5d83f948b
fix: log empty successful responses as request errors
5 дней назад
orangels
1a98836b94
fix: set bench max tokens default to 200k
5 дней назад
orangels
948e32d64a
feat: log request errors for debug
5 дней назад
orangels
f17c0af5d7
feat: tighten aime final answer parsing
5 дней назад
orangels
e0aca08f36
feat: align gpqa scoring with simple evals
5 дней назад
orangels
898d02f7ef
feat: record upstream error debug payloads
5 дней назад
orangels
5706ac8ee8
feat: add google gemini protocol
5 дней назад
orangels
6959de5eac
docs: add CLAUDE.md and update config.example.yaml
1 неделю назад
orangels
18cc7ff716
feat: add provider thinking parameters
1 неделю назад
orangels
ee13739c5e
feat: add raw response debug mode
1 неделю назад
orangels
60cc009c83
fix: support bare stream flag and cover SSE parsing
1 неделю назад
orangels
f21ab8bd4c
feat: add --max-tokens CLI param and correct_samples to bench reports
- Add --max-tokens parameter to bench subcommands (default 32768)
- Record up to 5 correct case samples in report JSON for output inspection
- Report max_tokens reflects actual value used instead of hardcoded 1024
1 неделю назад
orangels
4988a86654
feat: add progress bars to benchmark and RPM execution
Show real-time progress with elapsed time, completion count, and ETA
using indicatif. Progress bar clears on completion to keep final
output clean.
1 неделю назад
orangels
527b1e8fc7
feat: add params field to RPM and benchmark reports
Record CLI parameters (prompt, stream, duration, burst, concurrency,
probe_seconds, window_offset_ms) in report JSON so test conditions
can be fully reproduced from the report file alone.
1 неделю назад
orangels
e269f8f99d
docs: add end-to-end testing guide for RPM and benchmarks
1 неделю назад
orangels
fdb437a732
fix: correct streaming bugs and RPM diagnose gaps
- Move TTFT timer start before .send().await so it includes server
processing time (standard TTFT semantics)
- Add done flag to break outer while loop when stream ends
- Rewrite SseLineBuffer to use Vec<u8> with 1MB cap, fixing UTF-8
corruption on multi-byte chars split across chunks
- Add sliding_window_schedule probes to Diagnose mode so the
SlidingWindow inference branch is no longer dead code
- Move build_rpm_mode_plan after AppConfig::load to eliminate
window-boundary timing drift
1 неделю назад
orangels
20217b2250
feat: add streaming (SSE) support with TTFT metrics
Support streaming responses for both OpenAI and Anthropic protocols.
Adds --stream CLI flag, provider config stream field, shared SSE line
buffer, and TTFT (time to first token) percentile tracking alongside
existing TTLT latency metrics.
1 неделю назад
orangels
f5b9239167
docs: add Chinese usage guide
1 неделю назад
orangels
84943db37a
docs: document rpm limiter modes
1 неделю назад
orangels
2b9fae7ebd
feat: report rpm limiter mode details
1 неделю назад
orangels
0a96e11e98
feat: run rpm limiter modes
1 неделю назад
orangels
7c0d1e8af0
feat: add rpm mode schedules
1 неделю назад
orangels
f6b09d00a4
docs: plan rpm limiter modes
1 неделю назад
orangels
65ce102004
fix: resolve only selected provider env refs
1 неделю назад
orangels
4b433c25e6
fix: redact config tokens in debug output
1 неделю назад
orangels
31c41cb8ac
fix: satisfy final clippy checks
1 неделю назад
orangels
4e99ab6f37
docs: add usage guide
1 неделю назад
orangels
83edc789df
fix: make benchmark and rpm execution reliable
1 неделю назад
orangels
5d1163d635
feat: run benchmark and rpm tests
1 неделю назад
orangels
4a0e187e98
docs: specify benchmark report output
1 неделю назад
orangels
d0fed7b680
fix: harden protocol adapter errors
1 неделю назад
orangels
1e0926cfba
feat: add openai and anthropic request adapters
1 неделю назад
orangels
2064c69924
fix: tighten benchmark dataset helpers
1 неделю назад
orangels
67a4ea8af4
feat: add benchmark dataset fetch and prompts
1 неделю назад
orangels
690b815ae0
test: avoid global env mutation in config tests
1 неделю назад
orangels
b071c99ee6
feat: load yaml relay config
1 неделю назад
orangels
c0ebdea077
chore: initialize cli project skeleton
1 неделю назад
orangels
ea0b255547
docs: add implementation plan
1 неделю назад
orangels
61446dcf9f
docs: define benchmark dataset download flow
1 неделю назад