| Yazar | SHA1 | Mesaj | Tarih |
|---|---|---|---|
|
|
8f89828ca3 |
fix(user-migration): 禁止冲突用户非法合并
- 前端冲突处理页面禁用 cn_already_synced_to_ov 场景的 merge 操作 - 后端 resolve 接口拒绝将已存在 synced copy 的 CN 用户作为 merge 目标 - 补充对应控制器测试 Co-Authored-By: Codex <noreply@anthropic.com> |
2 ay önce |
|
|
2f7a71d61e |
fix(user-migration): 补齐主节点保护与导入校验
- 为 user-migrations 管理路由增加仅 master 节点可访问的保护 - 为导入创建用户补充用户名、显示名、邮箱长度校验及测试 - 调整测试环境部署脚本,自动写入 CN/OV 的 NODE_TYPE Co-Authored-By: Codex <noreply@anthropic.com> |
2 ay önce |
|
|
9aaa6baa4c |
merge: user migration selective batches into master
# Conflicts: # controller/user.go # web/src/i18n/locales/en.json # web/src/i18n/locales/zh-CN.json |
2 ay önce |
|
|
65e6a82d34 |
feat(user-migration): support selective batches and cancellation
Add candidate user selection and explicit_ids batch creation flow for overseas migration. Also add cancellable batches to release selected-user locks and extend backend, frontend, and e2e coverage for the new flow. Co-Authored-By: Codex <noreply@anthropic.com> |
2 ay önce |
|
|
ca3f8765bb |
test(user-migration): complete migration coverage and e2e flow
Co-Authored-By: Codex <noreply@anthropic.com> |
2 ay önce |
|
|
a72d76522f | fix(deploy): 部署脚本 tag 格式与 Makefile 对齐,加入 commit hash 和 dirty 标记 | 2 ay önce |
|
|
de993c5605 |
feat(docker): 镜像 tag 加入 commit hash,未提交代码标记 dirty
格式:{时间}-{分支}-{短commit}[-dirty]
示例:202606051500-master-e9d5d51b(干净)
202606051500-master-e9d5d51b-dirty(有未提交改动)
|
2 ay önce |
|
|
e9d5d51b37 |
fix(sync): 修复 Slave 端编辑同步用户时误报余额不可修改的问题
前端对同步用户用 SyncedQuota 替代 Quota 展示,但后端比较时 用的是数据库真实 Quota 字段,导致两者不同而触发拦截。 改为比较 SyncedQuota,只有真正修改额度时才拦截。 |
2 ay önce |
|
|
2544556f5d | fix(ratio): 缓存读取倍率为 0 时前端不显示,补充 extraText 说明 | 2 ay önce |
|
|
dce7631092 | docs(ratio): 缓存创建倍率 extraText 补充设置为 0 时隐藏说明 | 2 ay önce |
|
|
1c15ee996d |
feat(ratio): 升级未设置倍率模型编辑弹窗,对齐可视化编辑器
- 将操作列「高级比例」按钮改为编辑图标按钮,打开完整编辑弹窗 - 编辑弹窗新增定价模式切换(按量计费/按次计费) - 按量计费下支持按倍率设置和按价格设置两种子模式 - 按倍率:模型倍率、补全倍率 - 按价格:输入价格、输出价格($/1M tokens),自动计算倍率 - 高级比例区域:缓存读取、缓存创建、图片、音频输入、音频输出 - 修复 SubmitData 遗漏高级倍率字段保存的 bug(CacheRatio 等 5 个字段) - 清理死代码(未调用的转换函数、残留 console.log) |
2 ay önce |
|
|
182ab5ff83 |
refactor(ratio): move advanced ratios to edit modal in unset models page
- Remove advanced ratio columns from the table - Add 'advanced ratios' button in action column - Add edit modal with Form.InputNumber fields (same as visual editor) - Table stays clean, advanced settings only visible on click Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
1fdf0a3f47 |
feat(ratio): add advanced ratio fields to unset models editor
- Add cache/image/audio ratio columns to the table - Add advanced ratio fields to the add model modal - Update data init, submit, and addModel to handle new fields - Use formRef to read InputNumber values (same fix as visual editor) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
d36a686f6b |
fix(ratio): preserve 0 as valid value for advanced ratio fields
- Replace || with ?? in addOrUpdateModel to keep 0 values (0 || '' = '') - Add null check in SubmitData to avoid writing undefined - Fix formRef value reading to preserve 0 vs empty distinction Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
c824381642 |
fix(ratio): read advanced ratio values from form on submit
Form.InputNumber values were not synced to currentModel state via onChange, so the advanced ratio fields were always empty when saving. Now reads values directly from formRef in onOk handler. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
7b34fbc88a |
feat(pricing): hide cache creation price when ratio is 0
- Add cacheCreationRatio !== 0 check for row data - Add hasCacheCreationPricing check for column visibility - Models with cache read but no cache creation (e.g. Gemini) no longer show empty cache creation price column Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
a8c739e466 |
feat(ui): add extraText descriptions and fix placeholder for advanced ratios
- Add extraText for cache read ratio and cache creation ratio fields - Update placeholder format to match channel pricing convention - Add missing i18n translations for extraText and new placeholders Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
192eb70862 |
feat(save): serialize and save advanced ratio fields
- Add 5 new fields to output object - Convert model data to float and populate output fields - Serialize all fields to JSON in finalOutput - Submit 7 total option fields (2 existing + 5 new) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
91e648c530 |
feat(add): handle advanced ratios in addOrUpdateModel
- Add 5 advanced ratio fields to updatedModel object - Use empty string fallback for undefined values Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
194ebbaa56 |
feat(edit): populate advanced ratios in edit modal
- Add 5 advanced ratio fields to formRef.current.setValues - Ensure existing ratio data displays correctly in edit modal Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
2ca560ccef |
feat(ui): add advanced ratios form section
- Add Form.Section with 5 advanced ratio input fields
- Configure min={0}, step={0.01} for all fields
- Add placeholders showing default values
- Only show when pricingMode === 'per-token'
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2 ay önce |
|
|
0ea023037a |
feat(ratio): parse advanced ratio fields in data initialization
- Parse 5 new JSON fields from props.options - Extend modelNames Set to include keys from all ratio fields - Add 5 new fields to model data objects Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
dab78ff2c4 |
feat(i18n): add advanced ratio translations
- Add Chinese translations for 5 advanced ratio fields - Add English translations for 5 advanced ratio fields Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
a9fc624ff2 |
docs: add extended visual ratio settings implementation plan
- 9 detailed tasks covering i18n, data, UI, and testing - Each task broken into 2-5 minute steps with exact code - Complete validation and troubleshooting sections - Estimated 2.5 hours total implementation time Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
b699dda2ad |
docs: add extended visual ratio settings design spec
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
491e9b8ee4 |
fix(pricing): GetCompletionRatio 优先使用用户设置值而非硬编码默认值
修复 GetCompletionRatio 中硬编码默认值优先于用户设置值的 bug。 之前硬编码的 gpt-5 前缀匹配 (return 8, true) 会直接返回, 导致用户在后台设置的 completion_ratio 被忽略。 现在改为先查 completionRatioMap(用户设置),再查硬编码默认值。 |
2 ay önce |
|
|
7af60d3214 |
feat(pricing): 分组价格表格添加缓存读取和缓存创建价格列
- calculateModelPrice 新增缓存读取/创建价格计算逻辑 - ModelPricingTable 条件显示缓存价格列(仅 cache_ratio !== 1 时) - deploy.sh 添加镜像拉取超时(120s)和3次重试机制 |
2 ay önce |
|
|
3a78e9c93a | fix(user-migration): revalidate drift and verify synced quota | 2 ay önce |
|
|
36b47379f7 | test(user-migration): cover rescan and batch status regressions | 2 ay önce |
|
|
a406c94b93 | feat(user-migration): add root migration dashboard | 2 ay önce |
|
|
7282142316 | feat(user-migration): add root admin migration API | 2 ay önce |
|
|
2ecaa4f818 | feat(user-migration): add executor and verifier | 2 ay önce |
|
|
9e72c2f068 | feat(user-migration): add scan service with conflict analysis | 2 ay önce |
|
|
102b15ad41 | feat(user-migration): add internal migration endpoints and client | 2 ay önce |
|
|
38ba0dd24b | feat(user-migration): add imported user and oauth migration helpers | 2 ay önce |
|
|
26cb2273a4 | feat(user-migration): add batch item and quota grant models | 2 ay önce |
|
|
87a0f94975 |
refactor: remove user-channel-ratio feature
删除用户渠道倍率(UserChannelRatio)功能,清理全链路相关代码: 后端删除 model/controller/router 层实现,前端删除 UI 组件和计费展示逻辑。 同时恢复 Token 分组选择和价格侧边栏分组过滤器的注释代码。 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
0c279e6662 | docs: add overseas user migration design | 2 ay önce |
|
|
6ad0524172 |
test: add comprehensive pricing and channel selection tests
Add unit tests for group ratio, cross-group retry, and price calculation: - Group ratio CRUD and special ratio override (12 tests) - HandleGroupRatio with auto-group and special ratios (5 tests) - ModelPriceHelper with group ratio in price/ratio modes (3 tests) - Channel selection priority, groups, weighted LB (6 tests) - RetryParam and AutoGroup service logic (7 tests) Also restore the group pricing card in model detail SideSheet. |
2 ay önce |
|
|
052b41562a |
fix(pricing): restore group info on model cards
Uncomment the group ratio display that was hidden in
|
2 ay önce |
|
|
0c2ea777f2 |
fix: remove channel-pricing API calls from frontend
Backend endpoints were removed but frontend still called them causing 404s. ChannelPricingCard no longer fetches, channelPricingApi and pricingTagApi return empty stubs to keep dependent components stable. |
2 ay önce |
|
|
c64e9c0b40 |
chore: remove .agents from git tracking
Add .agents/ to .gitignore and untrack skill files. |
2 ay önce |
|
|
a88ab5914a |
chore: remove web/dist from git tracking
Revert .gitignore exceptions that accidentally tracked web/dist build artifacts. The directory should never be committed. |
2 ay önce |
|
|
7a2f584317 |
fix(claude): use gjson/sjson for reliable thinking.type replacement
Replace bytes.Replace with gjson.GetBytes+sjson.SetBytes in passthrough mode to correctly handle JSON whitespace variations and avoid silently sending empty body on read errors. Also add log search count limit to prevent slow COUNT queries on large log tables. |
2 ay önce |
|
|
8150993caf |
fix(admin): improve channel form inputs
Expose channel public_name in the edit modal and load a larger channel page when building model settings views. Co-Authored-By: Codex <noreply@anthropic.com> |
2 ay önce |
|
|
3aa4cdf2ec |
feat(models): add structured endpoint editor
Replace freeform endpoint JSON editing with a structured editor, keep advanced JSON mode, and make backend pricing parsing tolerate legacy array-form endpoint configs. Co-Authored-By: Codex <noreply@anthropic.com> |
2 ay önce |
|
|
481b43cfc5 |
fix(claude): send adaptive thinking type
Use adaptive thinking for Claude requests and rewrite passthrough bodies that still contain the legacy enabled type. Co-Authored-By: Codex <noreply@anthropic.com> |
2 ay önce |
|
|
23f9ed4ccd |
fix(redemption): block synced users from redeeming
Reject redemption requests from synced users, surface a localized API message, and cover both model- and controller-level paths with tests. Co-Authored-By: Codex <noreply@anthropic.com> |
2 ay önce |
|
|
2af3a9301d |
fix(pricing): remove stale channel pricing backend
Delete leftover channel pricing controllers and routes after removing channel pricing, and align relay price helpers with global pricing plus user channel ratio only. Co-Authored-By: Codex <noreply@anthropic.com> |
2 ay önce |
|
|
3a6dec8a6c |
Merge branch 'feat/remove-channel-pricing'
# Conflicts: # model/pricing_test.go |
2 ay önce |
|
|
6479883ddc |
fix: handle object-type arguments in Responses API stream
gpt-5.4 returns `arguments` as a JSON object for some tools (e.g. apply_patch, tool_search), but ResponsesOutput.Arguments was typed as string, causing json.Unmarshal to fail on every such SSE event. The failure silently dropped all output_item events and the response.completed event from the forwarded stream, so: - the client never received tool call content - usage/token counts could not be extracted (billed as 0) - client reported "stream closed before response.completed" Fix: change Arguments to json.RawMessage and add GetArguments() which normalises both forms to a plain string (unescapes JSON strings, returns raw bytes for objects). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
674b96298c | refactor(model): remove channel pricing | 2 ay önce |
|
|
415174a676 | test(model): switch pricing tests to global defaults | 2 ay önce |
|
|
7ecedd8e57 | fix: restore clean go test baseline | 2 ay önce |
|
|
3cf84775f2 |
feat: add channel metrics collector, fix search pagination and vendor sort order
- Register Prometheus ChannelCollector in GetMetrics() sync.Once block - Add MetricsMiddleware to /v1 and /v1beta relay router groups when metrics enabled - Remove public_name field from channel creation form and validation - Restore InvitationCard in topup page - Extract fetchSearchPage helper in useModelsData to eliminate ~50 lines of duplication; fix setSearching with finally block; use Axios params for URL encoding; fix missing vendor_counts update during paginated search - Fix vendor ordering in HomePricingFilters to follow vendorsMap sort_order; fix O(n²) Array.includes to use Set Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
2 ay önce |
|
|
6e81faa698 |
perf: lazy-read body in DoApiRequest for passthrough model mapping
DoApiRequest previously called io.ReadAll on every request body to apply passthrough model mapping, even when the feature was disabled. This caused unnecessary memory pressure for large request bodies (long conversations, multipart image uploads). Split the guard logic into needsPassthroughModelMapping() so that ReadAll is only called when all conditions are met (feature enabled + passthrough mode + model mapped). Non-passthrough requests now pass the original io.Reader directly to http.NewRequest without buffering. Rewrite tests to verify combined guard + modify behavior. |
2 ay önce |
|
|
6434a523a1 | feat: passthrough model mapping | 2 ay önce |
|
|
4584a94620 | Merge branch 'feat/custom-nav-link' | 2 ay önce |
|
|
d32050d081 | fix: restore login widget bundle and guard custom nav link | 2 ay önce |
|
|
54573991a4 |
feat: add customLink configuration UI in header nav settings
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
2 ay önce |
|
|
49c6755622 |
feat: add customLink to navigation hook with filtering logic
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
2 ay önce |
|
|
79f8a18cef | feat(widget): 重构登录组件为无 UI 的 headless SDK | 2 ay önce |
|
|
60a1403ae2 |
fix(widget): Dockerfile 添加 Widget 构建步骤
主前端构建后执行 widget 构建,确保 login-widget.js 被 embed 打包 |
2 ay önce |
|
|
aa15965467 |
fix: Dockerfile 使用华为云镜像源解决构建网络问题
移除 # syntax 指令,将 Docker Hub 基础镜像替换为华为云 SWR 镜像,解决 WSL 环境下无法直连 Docker Hub 的问题。 |
2 ay önce |
|
|
1c5e6d03d9 |
fix: 禁止修改同步用户余额
同步用户余额由主节点管理,从节点不应允许管理员修改其余额。 前端禁用额度输入和添加额度按钮,后端增加校验拦截。 |
2 ay önce |
|
|
fe62509b03 | feat(metrics): 重试循环和错误处理中采集指标 | 2 ay önce |
|
|
e51a66c5f9 | feat(metrics): postConsumeQuota 中采集 token/配额/延迟指标 | 2 ay önce |
|
|
123f0d95ae | feat(metrics): 注册 /metrics 端点 | 2 ay önce |
|
|
87628f5469 | feat(metrics): 渠道状态自定义 Collector | 2 ay önce |
|
|
67e78f2c44 | feat(metrics): Gin 中间件 - 活跃请求、延迟、状态码 | 2 ay önce |
|
|
9d4378dbb5 | feat(metrics): 错误分类函数及测试 | 2 ay önce |
|
|
688c5f8666 | feat(metrics): Prometheus 指标定义和注册 | 2 ay önce |
|
|
5afa4c48fc |
feat: 用户表新增注册时间字段 + 用户列表显示邮箱和注册时间
- model/user.go: 新增 CreatedAt 字段,Insert 时自动填充 - UsersColumnDefs.jsx: 用户列表新增邮箱、注册时间列 - makefile: 新增 build-widget target |
2 ay önce |
|
|
0c369eba51 |
docs: 添加登录 Widget 使用指南
API 参考、完整示例、主题配置、常见问题等 |
2 ay önce |
|
|
763cf86a7a |
fix(widget): 修复跨域 CORS 问题
- CORS 中间件改用 AllowOriginFunc 回显具体 origin,替代 AllowAllOrigins:* - userRoute 添加 CORS 中间件,登录接口支持跨域请求 - Widget fetch 移除 credentials: 'include',无需 Cookie |
2 ay önce |
|
|
de3b359dea |
feat(widget): 登录 Widget v1 — 密码登录,Shadow DOM 隔离
- 新增 web/widget/ 独立 Vite 构建,IIFE 格式打包 - Shadow DOM 样式隔离,支持 light/dark 主题 - 调用现有 /api/user/login 接口,零后端改动 - 构建产物 web/dist/static/login-widget.js (146KB) |
2 ay önce |
|
|
3c5b821584 |
docs: 添加登录 Widget 实施计划
6 个 Task:项目骨架 → 样式 → LoginForm → 入口 → 构建集成 → E2E 验证 |
2 ay önce |
|
|
08b89e6252 |
docs: 添加登录 Widget 设计文档
纯前端登录表单 Widget,通过 <script> 标签嵌入到第三方页面, 调用现有 /api/user/login 接口,零后端改动。 |
2 ay önce |
|
|
84eb6d6a88 |
feat: 端点格式限制功能 + 完善测试
当开启端点格式限制后,不同协议的渠道只能通过对应的端点访问: - Anthropic 渠道只能用 /v1/messages - Gemini 渠道只能用 /v1beta/models/* - OpenAI 系渠道只能用 /v1/chat/completions 等 - VertexAI/阿里云等支持多协议的渠道不受限 核心实现:EndpointFormatGuard 中间件从请求路径推导协议家族, 不依赖 gin context 中的 relay_format(因 group middleware 执行 时 context 中尚无此值)。 修复了 /v1/models/*path 和 /v1/engines/ 路径被错误归类为 openai 家族的 bug(实际是 Gemini 格式)。 单元测试 58 个,E2E 测试 15 个,覆盖: - 路径分类边界(含 Gemini relay 路径修复验证) - 渠道类型覆盖(含非 LLM 渠道透传) - 并发安全、运行时开关切换 - 错误消息 JSON 结构和内容验证 |
2 ay önce |
|
|
4c76265f32 |
test: 添加渠道选择删除 E2E 测试 + 清理临时截图 + 新增辅助脚本
- 新增 E2E 测试脚本 test/e2e_channel_removal.py,覆盖 Token 解析、 已删除路由验证、请求自动分发、channel_id 忽略、定价无默认渠道等 - 新增 anthropic_cache_test.sh 缓存测试脚本 - 新增本地开发辅助脚本 (dev.ps1, start-local.ps1 等) - 清理根目录下 9 个临时测试截图 |
3 ay önce |
|
|
c64afdc6c2 |
fix: 清理渠道选择删除后的代码质量问题
- router/api-router.go: 修复删除路由后的缩进异常 - model/channel_pricing.go: 清理 Insert/Update/Delete 中遗留空行 - controller/relay_test.go: newTestContext 接收 *testing.T 参数并调用 t.Helper() |
3 ay önce |
|
|
631e030411 |
test: 添加渠道选择功能删除后的验证测试
验证 parseTokenKey 不再解析 channelId(sk-abc123:42 中的 :42 保留为 key 一部分),shouldRetry/shouldRetryTaskRelay 不再检查 specific_channel_id,所有请求都能正常重试。 |
3 ay önce |
|
|
b49842d543 |
refactor: 清理 Distribute 中的冗余条件判断和缩进
- 删除 tautological 的 `if channel == nil` 外层判断(channel 在该位置永远为 nil) - 修正删除指定渠道分支后遗留的多余缩进(减少一级 tab) - 更新通道亲和性检查注释为简洁版 |
3 ay önce |
|
|
ea0d4e5cde | refactor: 完整删除前端渠道选择和默认通道 UI | 3 ay önce |
|
|
baed9754d0 |
refactor: 完整删除渠道选择功能后端代码
删除了用户通过 sk-{key}:{channelId} 指定渠道、Playground 下拉框选择渠道、
以及管理员设置默认渠道 (is_default) 的所有后端代码。
涉及修改:
- middleware/auth.go: 简化 parseTokenKey,移除 specific_channel_id 设置
- middleware/distributor.go: 移除指定渠道分发逻辑、默认通道检查、ModelRequest.ChannelId
- constant/context_key.go: 移除 ContextKeyTokenSpecificChannelId
- controller/playground_channels.go: 整个文件删除
- controller/relay.go: 移除 shouldRetry/shouldRetryTaskRelay 中的指定渠道跳过
- controller/channel.go: 移除 GetUserChannelsForBinding
- controller/channel_pricing.go: 移除 SetDefaultChannel/ClearDefaultChannel
- model/ability.go: 移除 GetModelChannelsForGroup
- model/channel.go: 移除 GetAllChannelsForBinding
- model/channel_pricing.go: 移除默认通道缓存及所有相关函数
- model/pricing.go: 移除 DefaultChannelName 字段
- router/api-router.go: 移除 4 条路由
- i18n: 移除 channel.id_format_error 和 distributor.invalid_channel_id
|
3 ay önce |
|
|
547ed7e307 | docs: 渠道选择功能完整删除设计方案 | 3 ay önce |
|
|
f1a68027f1 |
Revert "feat(cache): OpenAI→Claude 转换自动注入 prompt caching"
This reverts commit
|
3 ay önce |
|
|
ff14caed7c |
feat(cache): OpenAI→Claude 转换自动注入 prompt caching
- ClaudeRequest 新增 CacheControl 字段
- RequestOpenAI2ClaudeMessage 自动注入顶层 cache_control: {"type":"ephemeral"}
- System 复合内容和普通消息 content block 的 CacheControl 透传
- ParseContent 新增 []MediaContent 类型断言,修复透传路径
- 新增 8 个单元测试覆盖自动注入、透传、序列化等场景
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
3 ay önce |
| @@ -0,0 +1,15 @@ | |||
| # 本地开发环境变量,与 docker/dev/docker-compose.yml 配套 | |||
| # 用法(WSL / bash): | |||
| # source .env.dev && go run main.go | |||
| # 用法(PowerShell): | |||
| # $env:SQL_DSN="root:Lanqi123456@tcp(localhost:33306)/new-api"; $env:REDIS_CONN_STRING="redis://localhost:36379"; go run main.go | |||
| export SQL_DSN="root:Lanqi123456@tcp(localhost:33306)/new-api" | |||
| export REDIS_CONN_STRING="redis://localhost:36379" | |||
| export SESSION_SECRET="dev-secret-do-not-use-in-prod" | |||
| export SESSION_NAME="session_dev" | |||
| export DEBUG=1 | |||
| export ERROR_LOG_ENABLED=true | |||
| export BATCH_UPDATE_ENABLED=true | |||
| export TZ=Asia/Shanghai | |||
| export ATLASCLOUD_API_KEY=apikey-ba28bd46b2bf470aa8a6718db4b6839a | |||
| @@ -23,6 +23,7 @@ plans | |||
| docs/plans/ | |||
| CLAUDE.md | |||
| .claude | |||
| .agents/ | |||
| .worktrees/ | |||
| logs/ | |||
| docs/superpowers | |||
| @@ -1,6 +1,4 @@ | |||
| # syntax=docker/dockerfile:1 | |||
| FROM oven/bun:latest AS builder | |||
| FROM swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/oven/bun:latest AS builder | |||
| # 使用淘宝镜像加速 | |||
| ENV BUN_INSTALL_REGISTRY=https://registry.npmmirror.com | |||
| @@ -12,6 +10,7 @@ RUN --mount=type=cache,target=/root/.bun/install/cache bun install | |||
| COPY ./web . | |||
| COPY ./VERSION . | |||
| RUN DISABLE_ESLINT_PLUGIN='true' VITE_REACT_APP_VERSION=$(cat VERSION) bun run build | |||
| RUN cd widget && bun install && bun run build | |||
| FROM swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/library/golang:1.26.1-alpine AS builder2 | |||
| ENV GO111MODULE=on CGO_ENABLED=0 | |||
| @@ -30,7 +29,7 @@ COPY . . | |||
| COPY --from=builder /build/dist ./web/dist | |||
| RUN --mount=type=cache,target=/root/.cache/go-build go build -ldflags "-s -w -X 'github.com/QuantumNous/new-api/common.Version=$(cat VERSION)'" -o new-api | |||
| FROM debian:bookworm-slim | |||
| FROM swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/library/debian:bookworm-slim | |||
| # 使用阿里云镜像源 | |||
| RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list.d/debian.sources | |||
| @@ -0,0 +1,123 @@ | |||
| # Alan 用户 ClaudeCode 令牌 429 错误 upstream_id 列表 | |||
| # 用户: Alan (user_id=47), 令牌: ClaudeCode (token_id=14) | |||
| # 时间范围: 2026-05-19 23:47:19 ~ 2026-05-20 09:03:54 | |||
| # 总计: 112 条 429 错误,全部来自 channel_id=1 | |||
| # 模型分布: claude-opus-4.7 (90条), claude-haiku-4.5 (22条) | |||
| # | |||
| # 格式: upstream_id | model_name | time | |||
| # === claude-haiku-4.5-20251001 (22条) === | |||
| 8a4c62b4-1686-4f02-901e-7e7da6aac605 anthropic/claude-haiku-4.5-20251001 2026-05-20 09:00:12 | |||
| 259035f9-eb62-484a-9e7f-13a3885a6626 anthropic/claude-haiku-4.5-20251001 2026-05-20 09:00:17 | |||
| 72745e0f-6a18-4093-93ed-9d9d1149ef7a anthropic/claude-haiku-4.5-20251001 2026-05-20 09:00:20 | |||
| 2f6faaf6-5511-4e22-8ce2-044346c175dc anthropic/claude-haiku-4.5-20251001 2026-05-20 09:00:27 | |||
| ec6e5677-8422-457a-870f-485a7c297790 anthropic/claude-haiku-4.5-20251001 2026-05-20 09:00:36 | |||
| a2523ccf-7241-4a61-ae8a-1e1d1db20daf anthropic/claude-haiku-4.5-20251001 2026-05-20 09:00:50 | |||
| 429e40ab-058b-4999-be18-246d1d8a4926 anthropic/claude-haiku-4.5-20251001 2026-05-20 09:01:12 | |||
| de03173d-c6e4-4327-8339-45cfa734066b anthropic/claude-haiku-4.5-20251001 2026-05-20 09:01:54 | |||
| f01c38ab-a216-4da2-8823-e2107e54839d anthropic/claude-haiku-4.5-20251001 2026-05-20 09:02:33 | |||
| 9a7dbef1-dd8b-426b-a63f-cbbb5195682a anthropic/claude-haiku-4.5-20251001 2026-05-20 09:03:15 | |||
| 82a4bbf6-bc89-4617-8514-a4ac2cf5174f anthropic/claude-haiku-4.5-20251001 2026-05-20 09:03:54 | |||
| 40c0b99d-4c34-474c-a942-e2f42785646a anthropic/claude-haiku-4.5-20251001 2026-05-19 23:51:01 | |||
| 5fcf397a-1345-4174-9f92-a4f1e0607b1b anthropic/claude-haiku-4.5-20251001 2026-05-19 23:50:23 | |||
| b22f2e85-07f7-4ffa-89de-2fa9ba5bb6a7 anthropic/claude-haiku-4.5-20251001 2026-05-19 23:49:47 | |||
| c8f0f203-0253-41ae-b72e-80d8667dd017 anthropic/claude-haiku-4.5-20251001 2026-05-19 23:49:00 | |||
| 4ec13969-7f2a-41d6-83cf-b4f1829e8fae anthropic/claude-haiku-4.5-20251001 2026-05-19 23:48:22 | |||
| aa3dd5ef-c73f-4527-b4b6-0ded7d445d92 anthropic/claude-haiku-4.5-20251001 2026-05-19 23:48:01 | |||
| 0435b06b-c437-4bc2-9d8a-b86a41110520 anthropic/claude-haiku-4.5-20251001 2026-05-19 23:47:47 | |||
| b9c49d19-e532-4386-b932-e72c3964b10c anthropic/claude-haiku-4.5-20251001 2026-05-19 23:47:38 | |||
| 97bcded4-ce94-4ba5-833d-fd5f74675ec8 anthropic/claude-haiku-4.5-20251001 2026-05-19 23:47:31 | |||
| 38066d1a-0749-424b-99b6-3f9dafc344cc anthropic/claude-haiku-4.5-20251001 2026-05-19 23:47:25 | |||
| 0a3365ed-a54b-421d-8125-f384e6a18d5b anthropic/claude-haiku-4.5-20251001 2026-05-19 23:47:19 | |||
| # === claude-opus-4.7 (90条) === | |||
| 8937e45a-090a-4f92-baa3-802ebdbadf9f anthropic/claude-opus-4.7 2026-05-20 06:19:10 | |||
| 2577f038-cb35-4b9c-9039-aa1a34ceced5 anthropic/claude-opus-4.7 2026-05-20 06:19:30 | |||
| d6180a7a-4dda-489e-8f4e-ac32ecb8d960 anthropic/claude-opus-4.7 2026-05-20 06:19:48 | |||
| 9d105cfb-2bc8-4f54-80a8-03d8c02c0e4e anthropic/claude-opus-4.7 2026-05-20 06:20:10 | |||
| fb1ce184-b582-4ce8-877e-4ff9b8c86c50 anthropic/claude-opus-4.7 2026-05-20 06:20:42 | |||
| a3248024-75fb-431b-909b-6832f5f47800 anthropic/claude-opus-4.7 2026-05-20 06:21:13 | |||
| 5adca109-c90a-495c-8900-8168a0d0e1b3 anthropic/claude-opus-4.7 2026-05-20 06:21:54 | |||
| 3fc887f4-14a9-4ead-9473-de5f3befa5f4 anthropic/claude-opus-4.7 2026-05-20 06:22:45 | |||
| 9b385114-e385-4bf6-93f5-65d29d695ada anthropic/claude-opus-4.7 2026-05-20 06:23:37 | |||
| 12d1f1a4-963c-4710-aaa9-aedcec883d54 anthropic/claude-opus-4.7 2026-05-20 06:24:40 | |||
| dd70e7e1-5658-48c5-ae41-bcc3648e0f3f anthropic/claude-opus-4.7 2026-05-20 06:25:41 | |||
| 97e7aaa9-c797-44d0-b241-39f4eb8ab314 anthropic/claude-opus-4.7 2026-05-20 06:32:03 | |||
| dd3c3de5-bad7-4819-a736-0112589bc3a8 anthropic/claude-opus-4.7 2026-05-20 06:32:22 | |||
| a599b1b9-0c60-4219-a7ab-0a0f595c425a anthropic/claude-opus-4.7 2026-05-20 06:32:40 | |||
| d763ca6b-e38e-4d51-b6f2-d27d6816ad9a anthropic/claude-opus-4.7 2026-05-20 06:33:07 | |||
| 5131c19d-87f7-44f0-9fd7-70642c8fd1ec anthropic/claude-opus-4.7 2026-05-20 06:33:46 | |||
| 3515ded8-b04e-4af3-9509-6dab3e20a8a7 anthropic/claude-opus-4.7 2026-05-20 06:34:16 | |||
| 1ddcd66b-10be-4017-a9f1-2e678e95455a anthropic/claude-opus-4.7 2026-05-20 06:35:03 | |||
| 8425c233-7892-4dc9-b240-a4c7c9d7ece3 anthropic/claude-opus-4.7 2026-05-20 06:36:58 | |||
| 9690cf62-abbe-4e9d-ab8d-4e6a66ee384e anthropic/claude-opus-4.7 2026-05-20 06:39:16 | |||
| c16b6d6d-52d9-4512-b864-6b7610bd200f anthropic/claude-opus-4.7 2026-05-20 06:43:08 | |||
| 0b1e715b-7dbf-499c-815a-837a50860631 anthropic/claude-opus-4.7 2026-05-20 06:44:19 | |||
| 7223d596-9dde-40e8-898a-c9f8a28c94ca anthropic/claude-opus-4.7 2026-05-20 06:44:51 | |||
| cf39ec6b-c982-4c1d-9af7-8d0bc0a4a740 anthropic/claude-opus-4.7 2026-05-20 06:49:34 | |||
| 341e944e-0475-44e6-a316-d3ab69748d9d anthropic/claude-opus-4.7 2026-05-20 06:50:19 | |||
| 44b66588-71f0-45ba-add9-8326d89727a5 anthropic/claude-opus-4.7 2026-05-20 06:54:02 | |||
| 8fbf4a48-30cb-4928-95ac-d63570ac4e50 anthropic/claude-opus-4.7 2026-05-20 06:56:08 | |||
| ba638c4b-fe4b-49e0-af48-61338523a16a anthropic/claude-opus-4.7 2026-05-20 06:57:18 | |||
| 2ed8faa7-c444-43e3-936c-e4254c12a9e7 anthropic/claude-opus-4.7 2026-05-20 06:57:48 | |||
| 8d3f86c8-a736-4f47-8b48-d6de8f0aed5b anthropic/claude-opus-4.7 2026-05-20 07:03:48 | |||
| 9b7858f5-e763-4ecf-836a-b66ec78b6e01 anthropic/claude-opus-4.7 2026-05-20 07:04:41 | |||
| 9b4eaf36-651e-4348-b9a0-d85d19e941e3 anthropic/claude-opus-4.7 2026-05-20 07:05:06 | |||
| 2147af82-839e-4e84-820f-abeb23c71289 anthropic/claude-opus-4.7 2026-05-20 07:05:32 | |||
| 34ed7bb3-99db-46f0-992f-3479223c168d anthropic/claude-opus-4.7 2026-05-20 07:06:00 | |||
| 050bd83b-10bb-4eef-8ba6-2ba80802bac8 anthropic/claude-opus-4.7 2026-05-20 07:06:44 | |||
| 2686785e-d2d5-4418-a8aa-f162de8924b4 anthropic/claude-opus-4.7 2026-05-20 07:07:21 | |||
| 12c8fd4f-d492-4fe1-8290-3e66523fcf56 anthropic/claude-opus-4.7 2026-05-20 07:07:59 | |||
| f8231f97-4a77-49c9-aaf7-55c89ea170d6 anthropic/claude-opus-4.7 2026-05-20 07:08:56 | |||
| bb4f79e7-7c35-413a-8d17-5968399133a6 anthropic/claude-opus-4.7 2026-05-20 07:10:07 | |||
| a07961a7-41cd-41dd-9fce-49b244328a33 anthropic/claude-opus-4.7 2026-05-20 07:11:09 | |||
| 33d1d725-a065-49dd-a371-31983ccf58c8 anthropic/claude-opus-4.7 2026-05-20 07:12:06 | |||
| 8b5e2509-2512-4bf0-96ea-115e318143df anthropic/claude-opus-4.7 2026-05-20 07:15:13 | |||
| 51c579f4-80d1-43c1-897e-20c6dc69465d anthropic/claude-opus-4.7 2026-05-20 07:15:26 | |||
| 471666f2-2f73-46c5-bd76-13620b0fd3c9 anthropic/claude-opus-4.7 2026-05-20 07:15:47 | |||
| abc8755a-4d56-4d36-8ca6-62ed68d2e979 anthropic/claude-opus-4.7 2026-05-20 07:16:05 | |||
| dca16c52-2c99-43a9-8f9c-96d910beaba5 anthropic/claude-opus-4.7 2026-05-20 07:16:43 | |||
| daa220d4-a311-4c33-bacd-72265ccf83cb anthropic/claude-opus-4.7 2026-05-20 07:17:05 | |||
| 3474a1ed-6f7a-48aa-8213-c201fca65c91 anthropic/claude-opus-4.7 2026-05-20 07:17:52 | |||
| 8f0f37f4-58dd-4b45-a0b4-53bddf35d1fe anthropic/claude-opus-4.7 2026-05-20 07:18:39 | |||
| 6cec0036-6c27-4e59-91a1-097c3aad49ab anthropic/claude-opus-4.7 2026-05-20 07:19:27 | |||
| 6b08f0cb-dfb7-4745-abb4-79f05b8d7b76 anthropic/claude-opus-4.7 2026-05-20 07:22:13 | |||
| 6408a6be-22a9-4a48-9b23-944e8e2ebdbe anthropic/claude-opus-4.7 2026-05-20 07:22:30 | |||
| 76f1a72b-4c91-4ca5-a889-f64c0e641353 anthropic/claude-opus-4.7 2026-05-20 07:22:47 | |||
| 8ecf0adc-c60f-434d-91e3-c3ec3f3e47f9 anthropic/claude-opus-4.7 2026-05-20 07:29:48 | |||
| 63283fcf-84cd-4778-b138-be9155ed2773 anthropic/claude-opus-4.7 2026-05-20 07:30:09 | |||
| ae5c146f-127e-4cfa-a664-1496e25c9208 anthropic/claude-opus-4.7 2026-05-20 07:30:46 | |||
| f3cf76fc-3741-4525-b7a3-dbcaf07651d4 anthropic/claude-opus-4.7 2026-05-20 07:31:09 | |||
| 075f9154-f857-4b59-b774-b85f4cb81a60 anthropic/claude-opus-4.7 2026-05-20 07:31:37 | |||
| 873a7295-d47c-4e3d-af8f-502d4a7e6a36 anthropic/claude-opus-4.7 2026-05-20 07:32:43 | |||
| 083f7117-4539-4bbc-8585-a0d520c8ad6b anthropic/claude-opus-4.7 2026-05-20 07:33:27 | |||
| a12a28fe-50a0-4572-b23a-6f833939014c anthropic/claude-opus-4.7 2026-05-20 07:34:20 | |||
| 4c5b9328-e370-4ed9-835b-151701076447 anthropic/claude-opus-4.7 2026-05-20 07:35:20 | |||
| 3b9bb953-bdf3-4585-bc79-146a48d34a21 anthropic/claude-opus-4.7 2026-05-20 07:36:19 | |||
| ffabd406-f4a8-41be-be59-099e30613fc1 anthropic/claude-opus-4.7 2026-05-20 07:37:15 | |||
| 61411ecc-5abb-4c47-bc15-055f27a51374 anthropic/claude-opus-4.7 2026-05-20 07:38:09 | |||
| a898a6a4-3ee5-41d1-a1ef-2b47cd2d7501 anthropic/claude-opus-4.7 2026-05-20 07:38:23 | |||
| f93ef5b5-4401-45e7-8ed8-59280b7b7ceb anthropic/claude-opus-4.7 2026-05-20 07:38:34 | |||
| 3f736901-0b44-4a7b-b1ff-9abea5688b9c anthropic/claude-opus-4.7 2026-05-20 07:38:44 | |||
| 09b55670-82ba-49e1-aa9f-376538a6d778 anthropic/claude-opus-4.7 2026-05-20 07:38:56 | |||
| 8efd0a1a-2cf3-44e6-976e-f99b6efb850d anthropic/claude-opus-4.7 2026-05-20 07:39:14 | |||
| a2431acd-8f1c-4f54-ac48-0b9000629f21 anthropic/claude-opus-4.7 2026-05-20 07:39:40 | |||
| e98d8348-7ed1-4677-90ee-51d180b863a8 anthropic/claude-opus-4.7 2026-05-20 07:40:43 | |||
| 10eb0a90-082e-47f2-a52c-a9cb64b518f7 anthropic/claude-opus-4.7 2026-05-20 08:43:09 | |||
| 0057a591-3727-402a-851f-b1e8610e3fca anthropic/claude-opus-4.7 2026-05-20 08:43:34 | |||
| e3ded410-5c25-4160-a325-4b4f7a341c19 anthropic/claude-opus-4.7 2026-05-20 08:43:53 | |||
| b3e197b9-0277-4e96-a533-b976effa9d64 anthropic/claude-opus-4.7 2026-05-20 08:44:34 | |||
| 241e074f-c084-405f-a46a-c4184b99b90a anthropic/claude-opus-4.7 2026-05-20 08:45:20 | |||
| 583b0e49-93f9-4138-84e1-e9fc2c4abaa3 anthropic/claude-opus-4.7 2026-05-20 08:45:51 | |||
| f1381633-4701-4d2b-ab00-4e15ac1e4024 anthropic/claude-opus-4.7 2026-05-20 08:50:19 | |||
| 063bf354-28be-4e3e-9f5c-9f88c929acdc anthropic/claude-opus-4.7 2026-05-20 08:50:35 | |||
| d44bc890-2756-4cd5-8ee4-60ba0543c8c4 anthropic/claude-opus-4.7 2026-05-20 08:51:16 | |||
| 628f9ce9-8cfe-4966-a33b-1b2df10b10d8 anthropic/claude-opus-4.7 2026-05-20 08:52:26 | |||
| 8884ae5d-d22a-4ea4-9247-818c4b6be648 anthropic/claude-opus-4.7 2026-05-20 08:52:44 | |||
| b73a8fa5-4cb4-4481-90d8-33d485077a45 anthropic/claude-opus-4.7 2026-05-20 08:53:05 | |||
| d3594203-fc5a-4858-8f8b-7f81f2ee52d5 anthropic/claude-opus-4.7 2026-05-20 08:54:28 | |||
| 71c44932-cf4a-423e-a8cc-14d18fbd0e42 anthropic/claude-opus-4.7 2026-05-20 08:55:20 | |||
| 183be15a-ab5d-470a-87b2-10b0032b5bc1 anthropic/claude-opus-4.7 2026-05-20 08:56:11 | |||
| 1e24fa4b-aeae-42a0-89c7-a08982edcbfb anthropic/claude-opus-4.7 2026-05-20 08:57:00 | |||
| 33191ba1-8252-4d81-beb8-0c39877408bc anthropic/claude-opus-4.7 2026-05-20 08:58:01 | |||
| 0293f22d-5042-42a3-837f-21df0cd1b53b anthropic/claude-opus-4.7 2026-05-19 23:56:18 | |||
| @@ -0,0 +1,23 @@ | |||
| { | |||
| "contents": { | |||
| "role": "USER", | |||
| "parts": [ | |||
| { | |||
| "text": "Create a Tom and Jerry Poster." | |||
| } | |||
| ] | |||
| }, | |||
| "generationConfig": { | |||
| "responseModalities": [ | |||
| "IMAGE" | |||
| ], | |||
| "imageConfig": { | |||
| "aspectRatio": "16:9" | |||
| } | |||
| }, | |||
| "safetySettings": { | |||
| "method": "PROBABILITY", | |||
| "category": "HARM_CATEGORY_DANGEROUS_CONTENT", | |||
| "threshold": "BLOCK_MEDIUM_AND_ABOVE" | |||
| } | |||
| } | |||
| @@ -0,0 +1,64 @@ | |||
| package metrics | |||
| import ( | |||
| "fmt" | |||
| "github.com/prometheus/client_golang/prometheus" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/model" | |||
| ) | |||
| // ChannelCollector 从数据库查询渠道状态并暴露为 Prometheus Gauge | |||
| type ChannelCollector struct { | |||
| channelStatus *prometheus.Desc | |||
| } | |||
| // NewChannelCollector 创建渠道状态 Collector | |||
| func NewChannelCollector() *ChannelCollector { | |||
| return &ChannelCollector{ | |||
| channelStatus: prometheus.NewDesc( | |||
| "newapi_channel_status", | |||
| "Channel enabled status (1=enabled, 0=disabled)", | |||
| []string{"channel", "channel_type"}, | |||
| nil, | |||
| ), | |||
| } | |||
| } | |||
| func (c *ChannelCollector) Describe(ch chan<- *prometheus.Desc) { | |||
| ch <- c.channelStatus | |||
| } | |||
| func (c *ChannelCollector) Collect(ch chan<- prometheus.Metric) { | |||
| if model.DB == nil { | |||
| return | |||
| } | |||
| var channels []struct { | |||
| Id int | |||
| Type int | |||
| Status int | |||
| } | |||
| err := model.DB.Table("channels").Select("id, type, status").Find(&channels).Error | |||
| if err != nil { | |||
| return | |||
| } | |||
| for _, ch2 := range channels { | |||
| value := float64(0) | |||
| if ch2.Status == common.ChannelStatusEnabled { | |||
| value = 1 | |||
| } | |||
| m, err := prometheus.NewConstMetric( | |||
| c.channelStatus, | |||
| prometheus.GaugeValue, | |||
| value, | |||
| fmt.Sprintf("%d", ch2.Id), | |||
| fmt.Sprintf("%d", ch2.Type), | |||
| ) | |||
| if err == nil { | |||
| ch <- m | |||
| } | |||
| } | |||
| } | |||
| @@ -0,0 +1,93 @@ | |||
| package metrics | |||
| import ( | |||
| "testing" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/prometheus/client_golang/prometheus" | |||
| "github.com/stretchr/testify/assert" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/model" | |||
| ) | |||
| func setupCollectorTestDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, _ := db.DB() | |||
| sqlDB.SetMaxOpenConns(1) | |||
| origDB := model.DB | |||
| model.DB = db | |||
| common.UsingSQLite = true | |||
| common.RedisEnabled = false | |||
| require.NoError(t, db.AutoMigrate(&model.Channel{})) | |||
| t.Cleanup(func() { | |||
| model.DB = origDB | |||
| sqlDB.Close() | |||
| }) | |||
| return db | |||
| } | |||
| func TestChannelCollector_CollectsChannelStatus(t *testing.T) { | |||
| db := setupCollectorTestDB(t) | |||
| ch1 := model.Channel{Id: 1, Name: "openai-1", Type: 1, Status: common.ChannelStatusEnabled} | |||
| ch2 := model.Channel{Id: 2, Name: "claude-1", Type: 14, Status: common.ChannelStatusEnabled} | |||
| ch3 := model.Channel{Id: 3, Name: "gemini-1", Type: 24, Status: 3} | |||
| require.NoError(t, db.Create(&ch1).Error) | |||
| require.NoError(t, db.Create(&ch2).Error) | |||
| require.NoError(t, db.Create(&ch3).Error) | |||
| collector := NewChannelCollector() | |||
| reg := prometheus.NewRegistry() | |||
| reg.MustRegister(collector) | |||
| mfs, err := reg.Gather() | |||
| require.NoError(t, err) | |||
| var found bool | |||
| for _, mf := range mfs { | |||
| if mf.GetName() == "newapi_channel_status" { | |||
| found = true | |||
| metrics := mf.GetMetric() | |||
| assert.Len(t, metrics, 3) | |||
| statusMap := map[string]float64{} | |||
| for _, m := range metrics { | |||
| chId := "" | |||
| for _, label := range m.GetLabel() { | |||
| if label.GetName() == "channel" { | |||
| chId = label.GetValue() | |||
| } | |||
| } | |||
| statusMap[chId] = m.GetGauge().GetValue() | |||
| } | |||
| assert.Equal(t, float64(1), statusMap["1"]) | |||
| assert.Equal(t, float64(1), statusMap["2"]) | |||
| assert.Equal(t, float64(0), statusMap["3"]) | |||
| } | |||
| } | |||
| assert.True(t, found, "newapi_channel_status metric not found") | |||
| } | |||
| func TestChannelCollector_EmptyDB(t *testing.T) { | |||
| setupCollectorTestDB(t) | |||
| collector := NewChannelCollector() | |||
| reg := prometheus.NewRegistry() | |||
| reg.MustRegister(collector) | |||
| mfs, err := reg.Gather() | |||
| require.NoError(t, err) | |||
| for _, mf := range mfs { | |||
| assert.NotEqual(t, "newapi_channel_status", mf.GetName(), "should not have channel status with empty DB") | |||
| } | |||
| } | |||
| @@ -0,0 +1,53 @@ | |||
| package metrics | |||
| import ( | |||
| "strconv" | |||
| "strings" | |||
| ) | |||
| // ClassifyError 根据状态码和错误消息分类错误 | |||
| func ClassifyError(statusCode int, errMsg string) (string, string) { | |||
| // 超时类 | |||
| if strings.Contains(errMsg, "context deadline exceeded") { | |||
| return "timeout", "context_deadline_exceeded" | |||
| } | |||
| if strings.Contains(errMsg, "context canceled") { | |||
| return "timeout", "context_canceled" | |||
| } | |||
| // 配额/计费类 | |||
| if statusCode == 402 { | |||
| return "billing", "insufficient_quota" | |||
| } | |||
| if statusCode == 403 && strings.Contains(strings.ToLower(errMsg), "quota") { | |||
| return "billing", "quota_exceeded" | |||
| } | |||
| // 认证类 | |||
| if statusCode == 401 { | |||
| return "auth", "401" | |||
| } | |||
| if statusCode == 403 && !strings.Contains(strings.ToLower(errMsg), "quota") { | |||
| return "auth", "403" | |||
| } | |||
| // 速率限制 | |||
| if statusCode == 429 { | |||
| return "rate_limit", "429" | |||
| } | |||
| // 上游错误(5xx) | |||
| if statusCode >= 500 { | |||
| return "upstream", strconv.Itoa(statusCode) | |||
| } | |||
| if statusCode >= 400 { | |||
| return "other", strconv.Itoa(statusCode) | |||
| } | |||
| // 无状态码的未知错误 | |||
| if statusCode == 0 { | |||
| return "other", "unknown" | |||
| } | |||
| return "other", strconv.Itoa(statusCode) | |||
| } | |||
| @@ -0,0 +1,85 @@ | |||
| package metrics | |||
| import ( | |||
| "testing" | |||
| "github.com/stretchr/testify/assert" | |||
| ) | |||
| func TestClassifyError_RateLimitEmptyMsg(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(429, "") | |||
| assert.Equal(t, "rate_limit", errorType) | |||
| assert.Equal(t, "429", errorCode) | |||
| } | |||
| func TestClassifyError_UpstreamHTTP502(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(502, "") | |||
| assert.Equal(t, "upstream", errorType) | |||
| assert.Equal(t, "502", errorCode) | |||
| } | |||
| func TestClassifyError_UpstreamHTTP500(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(500, "") | |||
| assert.Equal(t, "upstream", errorType) | |||
| assert.Equal(t, "500", errorCode) | |||
| } | |||
| func TestClassifyError_Timeout(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(0, "context deadline exceeded") | |||
| assert.Equal(t, "timeout", errorType) | |||
| assert.Equal(t, "context_deadline_exceeded", errorCode) | |||
| } | |||
| func TestClassifyError_TimeoutCanceled(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(0, "context canceled") | |||
| assert.Equal(t, "timeout", errorType) | |||
| assert.Equal(t, "context_canceled", errorCode) | |||
| } | |||
| func TestClassifyError_AuthInvalidToken(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(401, "invalid api key") | |||
| assert.Equal(t, "auth", errorType) | |||
| assert.Equal(t, "401", errorCode) | |||
| } | |||
| func TestClassifyError_AuthForbidden(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(403, "") | |||
| assert.Equal(t, "auth", errorType) | |||
| assert.Equal(t, "403", errorCode) | |||
| } | |||
| func TestClassifyError_RateLimit(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(429, "rate limit exceeded") | |||
| assert.Equal(t, "rate_limit", errorType) | |||
| assert.Equal(t, "429", errorCode) | |||
| } | |||
| func TestClassifyError_BillingInsufficientQuota(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(402, "insufficient quota") | |||
| assert.Equal(t, "billing", errorType) | |||
| assert.Equal(t, "insufficient_quota", errorCode) | |||
| } | |||
| func TestClassifyError_BillingQuotaExceeded(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(403, "quota exceeded") | |||
| assert.Equal(t, "billing", errorType) | |||
| assert.Equal(t, "quota_exceeded", errorCode) | |||
| } | |||
| func TestClassifyError_Other(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(0, "something unexpected") | |||
| assert.Equal(t, "other", errorType) | |||
| assert.Equal(t, "unknown", errorCode) | |||
| } | |||
| func TestClassifyError_BadRequest(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(400, "") | |||
| assert.Equal(t, "other", errorType) | |||
| assert.Equal(t, "400", errorCode) | |||
| } | |||
| func TestClassifyError_EmptyBoth(t *testing.T) { | |||
| errorType, errorCode := ClassifyError(0, "") | |||
| assert.Equal(t, "other", errorType) | |||
| assert.Equal(t, "unknown", errorCode) | |||
| } | |||
| @@ -0,0 +1,119 @@ | |||
| package metrics | |||
| import ( | |||
| "net/http" | |||
| "os" | |||
| "sync" | |||
| "github.com/prometheus/client_golang/prometheus" | |||
| "github.com/prometheus/client_golang/prometheus/promhttp" | |||
| ) | |||
| var ( | |||
| globalMetrics *Metrics | |||
| globalOnce sync.Once | |||
| ) | |||
| // Metrics 包含所有 Prometheus 指标 | |||
| type Metrics struct { | |||
| RequestsTotal *prometheus.CounterVec | |||
| RequestErrorsTotal *prometheus.CounterVec | |||
| RequestRetriesTotal *prometheus.CounterVec | |||
| TokensTotal *prometheus.CounterVec | |||
| QuotaConsumedTotal *prometheus.CounterVec | |||
| RequestDuration *prometheus.HistogramVec | |||
| UpstreamDuration *prometheus.HistogramVec | |||
| FirstTokenDuration *prometheus.HistogramVec | |||
| ActiveRequests *prometheus.GaugeVec | |||
| } | |||
| // NewMetrics 创建并注册所有指标到给定 Registry | |||
| func NewMetrics(reg prometheus.Registerer) *Metrics { | |||
| m := &Metrics{ | |||
| RequestsTotal: prometheus.NewCounterVec(prometheus.CounterOpts{ | |||
| Name: "newapi_requests_total", | |||
| Help: "Total number of API requests", | |||
| }, []string{"channel", "model", "code", "is_stream", "relay_mode"}), | |||
| RequestErrorsTotal: prometheus.NewCounterVec(prometheus.CounterOpts{ | |||
| Name: "newapi_request_errors_total", | |||
| Help: "Total number of request errors", | |||
| }, []string{"channel", "model", "error_type", "error_code"}), | |||
| RequestRetriesTotal: prometheus.NewCounterVec(prometheus.CounterOpts{ | |||
| Name: "newapi_request_retries_total", | |||
| Help: "Total number of request retries", | |||
| }, []string{"channel", "model"}), | |||
| TokensTotal: prometheus.NewCounterVec(prometheus.CounterOpts{ | |||
| Name: "newapi_tokens_total", | |||
| Help: "Total tokens used", | |||
| }, []string{"channel", "model", "type"}), | |||
| QuotaConsumedTotal: prometheus.NewCounterVec(prometheus.CounterOpts{ | |||
| Name: "newapi_quota_consumed_total", | |||
| Help: "Total quota consumed", | |||
| }, []string{"channel", "model", "billing_source"}), | |||
| RequestDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{ | |||
| Name: "newapi_request_duration_seconds", | |||
| Help: "End-to-end request duration in seconds", | |||
| Buckets: []float64{0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30, 60, 120}, | |||
| }, []string{"channel", "model", "is_stream", "relay_mode"}), | |||
| UpstreamDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{ | |||
| Name: "newapi_upstream_duration_seconds", | |||
| Help: "Upstream response duration in seconds", | |||
| Buckets: []float64{0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30, 60, 120}, | |||
| }, []string{"channel", "model"}), | |||
| FirstTokenDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{ | |||
| Name: "newapi_first_token_duration_seconds", | |||
| Help: "Time to first token in seconds (streaming only)", | |||
| Buckets: []float64{0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10}, | |||
| }, []string{"channel", "model"}), | |||
| ActiveRequests: prometheus.NewGaugeVec(prometheus.GaugeOpts{ | |||
| Name: "newapi_active_requests", | |||
| Help: "Current number of active requests", | |||
| }, []string{"model"}), | |||
| } | |||
| reg.MustRegister( | |||
| m.RequestsTotal, | |||
| m.RequestErrorsTotal, | |||
| m.RequestRetriesTotal, | |||
| m.TokensTotal, | |||
| m.QuotaConsumedTotal, | |||
| m.RequestDuration, | |||
| m.UpstreamDuration, | |||
| m.FirstTokenDuration, | |||
| m.ActiveRequests, | |||
| ) | |||
| return m | |||
| } | |||
| // GetMetrics 返回全局 Metrics 实例 | |||
| func GetMetrics() *Metrics { | |||
| globalOnce.Do(func() { | |||
| if os.Getenv("METRICS_ENABLED") == "false" { | |||
| globalMetrics = NewMetrics(prometheus.NewPedanticRegistry()) | |||
| return | |||
| } | |||
| globalMetrics = NewMetrics(prometheus.DefaultRegisterer) | |||
| prometheus.MustRegister(NewChannelCollector()) | |||
| }) | |||
| return globalMetrics | |||
| } | |||
| // IsEnabled 返回 metrics 是否启用 | |||
| func IsEnabled() bool { | |||
| return os.Getenv("METRICS_ENABLED") != "false" | |||
| } | |||
| // NewMetricsHandler 返回默认 registry 的 /metrics HTTP handler | |||
| func NewMetricsHandler() http.Handler { | |||
| return promhttp.Handler() | |||
| } | |||
| // NewMetricsHandlerWithRegistry 返回指定 registry 的 /metrics HTTP handler | |||
| func NewMetricsHandlerWithRegistry(reg *prometheus.Registry) http.Handler { | |||
| return promhttp.HandlerFor(reg, promhttp.HandlerOpts{}) | |||
| } | |||
| @@ -0,0 +1,242 @@ | |||
| package metrics | |||
| import ( | |||
| "net/http/httptest" | |||
| "testing" | |||
| "time" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/prometheus/client_golang/prometheus" | |||
| "github.com/prometheus/client_golang/prometheus/testutil" | |||
| "github.com/stretchr/testify/assert" | |||
| ) | |||
| func newTestRegistry() *prometheus.Registry { | |||
| return prometheus.NewRegistry() | |||
| } | |||
| func TestRequestsTotalCounter(t *testing.T) { | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| m.RequestsTotal.WithLabelValues("1", "gpt-4", "200", "true", "chat").Inc() | |||
| m.RequestsTotal.WithLabelValues("1", "gpt-4", "200", "true", "chat").Inc() | |||
| m.RequestsTotal.WithLabelValues("2", "claude-3", "429", "false", "chat").Inc() | |||
| assert.Equal(t, float64(2), testutil.ToFloat64(m.RequestsTotal.WithLabelValues("1", "gpt-4", "200", "true", "chat"))) | |||
| assert.Equal(t, float64(1), testutil.ToFloat64(m.RequestsTotal.WithLabelValues("2", "claude-3", "429", "false", "chat"))) | |||
| } | |||
| func TestRequestErrorsTotalCounter(t *testing.T) { | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| m.RequestErrorsTotal.WithLabelValues("1", "gpt-4", "upstream", "429").Inc() | |||
| m.RequestErrorsTotal.WithLabelValues("1", "gpt-4", "upstream", "429").Inc() | |||
| m.RequestErrorsTotal.WithLabelValues("2", "claude-3", "timeout", "context_deadline_exceeded").Inc() | |||
| assert.Equal(t, float64(2), testutil.ToFloat64(m.RequestErrorsTotal.WithLabelValues("1", "gpt-4", "upstream", "429"))) | |||
| assert.Equal(t, float64(1), testutil.ToFloat64(m.RequestErrorsTotal.WithLabelValues("2", "claude-3", "timeout", "context_deadline_exceeded"))) | |||
| } | |||
| func TestRequestRetriesTotalCounter(t *testing.T) { | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| m.RequestRetriesTotal.WithLabelValues("1", "gpt-4").Inc() | |||
| m.RequestRetriesTotal.WithLabelValues("1", "gpt-4").Inc() | |||
| assert.Equal(t, float64(2), testutil.ToFloat64(m.RequestRetriesTotal.WithLabelValues("1", "gpt-4"))) | |||
| } | |||
| func TestTokensTotalCounter(t *testing.T) { | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| m.TokensTotal.WithLabelValues("1", "gpt-4", "prompt").Add(100) | |||
| m.TokensTotal.WithLabelValues("1", "gpt-4", "completion").Add(50) | |||
| m.TokensTotal.WithLabelValues("1", "gpt-4", "cache").Add(20) | |||
| assert.Equal(t, float64(100), testutil.ToFloat64(m.TokensTotal.WithLabelValues("1", "gpt-4", "prompt"))) | |||
| assert.Equal(t, float64(50), testutil.ToFloat64(m.TokensTotal.WithLabelValues("1", "gpt-4", "completion"))) | |||
| assert.Equal(t, float64(20), testutil.ToFloat64(m.TokensTotal.WithLabelValues("1", "gpt-4", "cache"))) | |||
| } | |||
| func TestQuotaConsumedTotalCounter(t *testing.T) { | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| m.QuotaConsumedTotal.WithLabelValues("1", "gpt-4", "wallet").Add(5000) | |||
| assert.Equal(t, float64(5000), testutil.ToFloat64(m.QuotaConsumedTotal.WithLabelValues("1", "gpt-4", "wallet"))) | |||
| } | |||
| func TestRequestDurationHistogram(t *testing.T) { | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| m.RequestDuration.WithLabelValues("1", "gpt-4", "true", "chat").Observe(1.5) | |||
| m.RequestDuration.WithLabelValues("1", "gpt-4", "true", "chat").Observe(2.5) | |||
| // 验证 histogram metric family 被注册且有数据 | |||
| assert.Equal(t, 1, testutil.CollectAndCount(m.RequestDuration)) | |||
| } | |||
| func TestUpstreamDurationHistogram(t *testing.T) { | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| m.UpstreamDuration.WithLabelValues("1", "gpt-4").Observe(0.5) | |||
| assert.Equal(t, 1, testutil.CollectAndCount(m.UpstreamDuration)) | |||
| } | |||
| func TestFirstTokenDurationHistogram(t *testing.T) { | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| m.FirstTokenDuration.WithLabelValues("1", "gpt-4").Observe(0.1) | |||
| assert.Equal(t, 1, testutil.CollectAndCount(m.FirstTokenDuration)) | |||
| } | |||
| func TestActiveRequestsGauge(t *testing.T) { | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| m.ActiveRequests.WithLabelValues("gpt-4").Inc() | |||
| m.ActiveRequests.WithLabelValues("gpt-4").Inc() | |||
| m.ActiveRequests.WithLabelValues("claude-3").Inc() | |||
| assert.Equal(t, float64(2), testutil.ToFloat64(m.ActiveRequests.WithLabelValues("gpt-4"))) | |||
| assert.Equal(t, float64(1), testutil.ToFloat64(m.ActiveRequests.WithLabelValues("claude-3"))) | |||
| m.ActiveRequests.WithLabelValues("gpt-4").Dec() | |||
| assert.Equal(t, float64(1), testutil.ToFloat64(m.ActiveRequests.WithLabelValues("gpt-4"))) | |||
| } | |||
| func TestMetricsDisabledIsNoOp(t *testing.T) { | |||
| // 当 METRICS_ENABLED=false 时,使用独立 registry(Discard) | |||
| reg := prometheus.NewPedanticRegistry() | |||
| m := NewMetrics(reg) | |||
| // 这些调用不应该 panic | |||
| m.RequestsTotal.WithLabelValues("1", "gpt-4", "200", "true", "chat").Inc() | |||
| m.ActiveRequests.WithLabelValues("gpt-4").Inc() | |||
| m.RequestDuration.WithLabelValues("1", "gpt-4", "true", "chat").Observe(1.0) | |||
| } | |||
| func TestMetricsMiddleware_ActiveRequests(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| r := gin.New() | |||
| r.Use(MetricsMiddleware(m)) | |||
| blockCh := make(chan struct{}) | |||
| r.GET("/test", func(c *gin.Context) { | |||
| <-blockCh | |||
| c.Status(200) | |||
| }) | |||
| go func() { | |||
| req := httptest.NewRequest("GET", "/test", nil) | |||
| w := httptest.NewRecorder() | |||
| r.ServeHTTP(w, req) | |||
| }() | |||
| go func() { | |||
| req := httptest.NewRequest("GET", "/test", nil) | |||
| w := httptest.NewRecorder() | |||
| r.ServeHTTP(w, req) | |||
| }() | |||
| time.Sleep(100 * time.Millisecond) | |||
| assert.Equal(t, float64(2), testutil.ToFloat64(m.ActiveRequests.WithLabelValues(""))) | |||
| close(blockCh) | |||
| time.Sleep(100 * time.Millisecond) | |||
| assert.Equal(t, float64(0), testutil.ToFloat64(m.ActiveRequests.WithLabelValues(""))) | |||
| } | |||
| func TestMetricsMiddleware_RequestsTotalByCode(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| r := gin.New() | |||
| r.Use(MetricsMiddleware(m)) | |||
| r.GET("/200", func(c *gin.Context) { c.Status(200) }) | |||
| r.GET("/400", func(c *gin.Context) { c.Status(400) }) | |||
| r.GET("/500", func(c *gin.Context) { c.Status(500) }) | |||
| for i := 0; i < 3; i++ { | |||
| req := httptest.NewRequest("GET", "/200", nil) | |||
| w := httptest.NewRecorder() | |||
| r.ServeHTTP(w, req) | |||
| } | |||
| req := httptest.NewRequest("GET", "/400", nil) | |||
| w := httptest.NewRecorder() | |||
| r.ServeHTTP(w, req) | |||
| req = httptest.NewRequest("GET", "/500", nil) | |||
| w = httptest.NewRecorder() | |||
| r.ServeHTTP(w, req) | |||
| assert.Equal(t, float64(3), testutil.ToFloat64(m.RequestsTotal.WithLabelValues("", "", "200", "", ""))) | |||
| assert.Equal(t, float64(1), testutil.ToFloat64(m.RequestsTotal.WithLabelValues("", "", "400", "", ""))) | |||
| assert.Equal(t, float64(1), testutil.ToFloat64(m.RequestsTotal.WithLabelValues("", "", "500", "", ""))) | |||
| } | |||
| func TestMetricsMiddleware_RequestDuration(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| reg := newTestRegistry() | |||
| m := NewMetrics(reg) | |||
| r := gin.New() | |||
| r.Use(MetricsMiddleware(m)) | |||
| r.GET("/slow", func(c *gin.Context) { | |||
| time.Sleep(100 * time.Millisecond) | |||
| c.Status(200) | |||
| }) | |||
| req := httptest.NewRequest("GET", "/slow", nil) | |||
| w := httptest.NewRecorder() | |||
| r.ServeHTTP(w, req) | |||
| assert.Equal(t, 1, testutil.CollectAndCount(m.RequestDuration)) | |||
| } | |||
| func TestMetricsEndpoint_Format(t *testing.T) { | |||
| handler := NewMetricsHandler() | |||
| req := httptest.NewRequest("GET", "/metrics", nil) | |||
| w := httptest.NewRecorder() | |||
| handler.ServeHTTP(w, req) | |||
| assert.Equal(t, 200, w.Code) | |||
| assert.Contains(t, w.Header().Get("Content-Type"), "text/plain") | |||
| } | |||
| func TestMetricsEndpoint_WithData(t *testing.T) { | |||
| reg := newTestRegistry() | |||
| _ = NewMetrics(reg) | |||
| handler := NewMetricsHandlerWithRegistry(reg) | |||
| req := httptest.NewRequest("GET", "/metrics", nil) | |||
| w := httptest.NewRecorder() | |||
| handler.ServeHTTP(w, req) | |||
| assert.Equal(t, 200, w.Code) | |||
| } | |||
| func TestMetricsEndpoint_EmptyNoPanic(t *testing.T) { | |||
| handler := NewMetricsHandler() | |||
| req := httptest.NewRequest("GET", "/metrics", nil) | |||
| w := httptest.NewRecorder() | |||
| handler.ServeHTTP(w, req) | |||
| assert.Equal(t, 200, w.Code) | |||
| } | |||
| @@ -0,0 +1,28 @@ | |||
| package metrics | |||
| import ( | |||
| "strconv" | |||
| "time" | |||
| "github.com/gin-gonic/gin" | |||
| ) | |||
| // MetricsMiddleware 是一个 Gin 中间件,采集活跃请求数、请求延迟和状态码。 | |||
| func MetricsMiddleware(m *Metrics) gin.HandlerFunc { | |||
| return func(c *gin.Context) { | |||
| start := time.Now() | |||
| m.ActiveRequests.WithLabelValues("").Inc() | |||
| defer func() { | |||
| m.ActiveRequests.WithLabelValues("").Dec() | |||
| elapsed := time.Since(start).Seconds() | |||
| status := strconv.Itoa(c.Writer.Status()) | |||
| m.RequestsTotal.WithLabelValues("", "", status, "", "").Inc() | |||
| m.RequestDuration.WithLabelValues("", "", "", "").Observe(elapsed) | |||
| }() | |||
| c.Next() | |||
| } | |||
| } | |||
| @@ -15,7 +15,6 @@ const ( | |||
| ContextKeyTokenKey ContextKey = "token_key" | |||
| ContextKeyTokenId ContextKey = "token_id" | |||
| ContextKeyTokenGroup ContextKey = "token_group" | |||
| ContextKeyTokenSpecificChannelId ContextKey = "specific_channel_id" | |||
| ContextKeyTokenModelLimitEnabled ContextKey = "token_model_limit_enabled" | |||
| ContextKeyTokenModelLimit ContextKey = "token_model_limit" | |||
| ContextKeyTokenCrossGroupRetry ContextKey = "token_cross_group_retry" | |||
| @@ -38,6 +37,8 @@ const ( | |||
| ContextKeyChannelMultiKeyIndex ContextKey = "channel_multi_key_index" | |||
| ContextKeyChannelKey ContextKey = "channel_key" | |||
| ContextKeyRelayFormat ContextKey = "relay_format" | |||
| ContextKeyAutoGroup ContextKey = "auto_group" | |||
| ContextKeyAutoGroupIndex ContextKey = "auto_group_index" | |||
| ContextKeyAutoGroupRetryIndex ContextKey = "auto_group_retry_index" | |||
| @@ -2106,30 +2106,3 @@ func OllamaVersion(c *gin.Context) { | |||
| }) | |||
| } | |||
| // GetUserChannelsForBinding 获取用户可用于绑定的渠道列表 | |||
| // 只返回 id、name、type 和 remark,不包含敏感信息(如key) | |||
| func GetUserChannelsForBinding(c *gin.Context) { | |||
| channels, err := model.GetAllChannelsForBinding() | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| // 构建返回结果 | |||
| result := make([]gin.H, 0, len(channels)) | |||
| for _, ch := range channels { | |||
| result = append(result, gin.H{ | |||
| "id": ch.Id, | |||
| "name": ch.Name, | |||
| "public_name": ch.PublicName, | |||
| "type": ch.Type, | |||
| "remark": ch.Remark, | |||
| }) | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{ | |||
| "success": true, | |||
| "message": "", | |||
| "data": result, | |||
| }) | |||
| } | |||
| @@ -1,406 +0,0 @@ | |||
| package controller | |||
| import ( | |||
| "fmt" | |||
| "strconv" | |||
| "strings" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/model" | |||
| "github.com/QuantumNous/new-api/setting/ratio_setting" | |||
| "github.com/gin-gonic/gin" | |||
| ) | |||
| // GetAllChannelPricing 获取所有渠道定价(分页) | |||
| func GetAllChannelPricing(c *gin.Context) { | |||
| page, _ := strconv.Atoi(c.DefaultQuery("p", "1")) | |||
| pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "10")) | |||
| offset := (page - 1) * pageSize | |||
| list, total, err := model.GetAllChannelPricing(offset, pageSize) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.ApiSuccess(c, gin.H{ | |||
| "page": page, | |||
| "page_size": pageSize, | |||
| "total": total, | |||
| "items": list, | |||
| }) | |||
| } | |||
| // GetChannelPricingByModel 获取指定模型的所有渠道定价 | |||
| func GetChannelPricingByModel(c *gin.Context) { | |||
| modelName := c.Param("name") | |||
| if modelName == "" { | |||
| common.ApiErrorMsg(c, "model name is required") | |||
| return | |||
| } | |||
| list, err := model.GetChannelPricingByModel(modelName) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.ApiSuccess(c, list) | |||
| } | |||
| // CreateChannelPricingRequest 创建渠道定价请求 | |||
| type CreateChannelPricingRequest struct { | |||
| Id int `json:"id"` | |||
| ModelName string `json:"model_name" binding:"required"` | |||
| ChannelId int `json:"channel_id" binding:"required"` | |||
| QuotaType int `json:"quota_type"` | |||
| ModelRatio float64 `json:"model_ratio"` | |||
| CompletionRatio float64 `json:"completion_ratio"` | |||
| ModelPrice float64 `json:"model_price"` | |||
| TagIds string `json:"tag_ids"` | |||
| CacheRatio float64 `json:"cache_ratio"` | |||
| CacheCreationRatio float64 `json:"cache_creation_ratio"` | |||
| ImageRatio float64 `json:"image_ratio"` | |||
| AudioRatio float64 `json:"audio_ratio"` | |||
| AudioCompletionRatio float64 `json:"audio_completion_ratio"` | |||
| } | |||
| // applyRequest 将请求字段应用到 ChannelPricing | |||
| func applyRequestFields(cp *model.ChannelPricing, req *CreateChannelPricingRequest) { | |||
| cp.ApplyFields(req.QuotaType, req.ModelRatio, req.CompletionRatio, req.ModelPrice, req.TagIds, | |||
| req.CacheRatio, req.CacheCreationRatio, req.ImageRatio, req.AudioRatio, req.AudioCompletionRatio) | |||
| } | |||
| // CreateChannelPricing 创建或更新渠道定价 | |||
| func CreateChannelPricing(c *gin.Context) { | |||
| var req CreateChannelPricingRequest | |||
| if err := c.ShouldBindJSON(&req); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| if req.CacheRatio < 0 || req.CacheCreationRatio < 0 || req.ImageRatio < 0 || req.AudioRatio < 0 || req.AudioCompletionRatio < 0 { | |||
| common.ApiErrorMsg(c, "ratio values must be >= 0") | |||
| return | |||
| } | |||
| // 检查是否已存在 | |||
| existing, _ := model.GetChannelPricing(req.ModelName, req.ChannelId) | |||
| if existing != nil { | |||
| applyRequestFields(existing, &req) | |||
| if err := existing.Update(); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.SysLog(fmt.Sprintf("[ChannelPricing] updated: id=%d model=%s channel=%d", existing.Id, existing.ModelName, existing.ChannelId)) | |||
| common.ApiSuccess(c, existing) | |||
| return | |||
| } | |||
| // 创建 | |||
| cp := &model.ChannelPricing{ | |||
| ModelName: req.ModelName, | |||
| ChannelId: req.ChannelId, | |||
| } | |||
| applyRequestFields(cp, &req) | |||
| if err := cp.Insert(); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.SysLog(fmt.Sprintf("[ChannelPricing] created: model=%s channel=%d quotaType=%d modelRatio=%.4f completionRatio=%.4f modelPrice=%.4f cacheRatio=%.4f cacheCreationRatio=%.4f imageRatio=%.4f audioRatio=%.4f audioCompletionRatio=%.4f", | |||
| req.ModelName, req.ChannelId, req.QuotaType, req.ModelRatio, req.CompletionRatio, req.ModelPrice, | |||
| req.CacheRatio, req.CacheCreationRatio, req.ImageRatio, req.AudioRatio, req.AudioCompletionRatio)) | |||
| common.ApiSuccess(c, cp) | |||
| } | |||
| // BatchCreateChannelPricingRequest 批量创建请求 | |||
| type BatchCreateChannelPricingRequest struct { | |||
| Items []*CreateChannelPricingRequest `json:"items" binding:"required"` | |||
| } | |||
| // BatchCreateChannelPricing 批量创建或更新渠道定价 | |||
| func BatchCreateChannelPricing(c *gin.Context) { | |||
| var req BatchCreateChannelPricingRequest | |||
| if err := c.ShouldBindJSON(&req); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| pricings := make([]*model.ChannelPricing, 0, len(req.Items)) | |||
| for _, item := range req.Items { | |||
| cp := &model.ChannelPricing{ | |||
| ModelName: item.ModelName, | |||
| ChannelId: item.ChannelId, | |||
| } | |||
| applyRequestFields(cp, item) | |||
| pricings = append(pricings, cp) | |||
| } | |||
| // 使用事务逐个处理(GORM 的批量 upsert 在不同数据库表现不一致) | |||
| for _, cp := range pricings { | |||
| existing, _ := model.GetChannelPricing(cp.ModelName, cp.ChannelId) | |||
| if existing != nil { | |||
| existing.ApplyFields(cp.QuotaType, cp.ModelRatio, cp.CompletionRatio, cp.ModelPrice, cp.TagIds, | |||
| cp.CacheRatio, cp.CacheCreationRatio, cp.ImageRatio, cp.AudioRatio, cp.AudioCompletionRatio) | |||
| if err := existing.Update(); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| } else { | |||
| if err := cp.Insert(); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| } | |||
| } | |||
| common.ApiSuccess(c, gin.H{"affected": len(pricings)}) | |||
| } | |||
| // DeleteChannelPricing 删除渠道定价 | |||
| func DeleteChannelPricing(c *gin.Context) { | |||
| idStr := c.Param("id") | |||
| id, err := strconv.Atoi(idStr) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| cp := &model.ChannelPricing{Id: id} | |||
| if err := cp.Delete(); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.SysLog(fmt.Sprintf("[ChannelPricing] deleted: id=%d", id)) | |||
| common.ApiSuccess(c, nil) | |||
| } | |||
| // CopyGlobalPricingRequest 复制全局定价请求 | |||
| type CopyGlobalPricingRequest struct { | |||
| Overwrite bool `json:"overwrite"` // 是否覆盖已存在的渠道定价 | |||
| } | |||
| // CopyGlobalPricing 复制全局定价到指定渠道 | |||
| // 从 ratio_setting 读取全局定价信息,复制到 channel_pricing 表 | |||
| func CopyGlobalPricing(c *gin.Context) { | |||
| channelIdStr := c.Param("channel_id") | |||
| channelId, err := strconv.Atoi(channelIdStr) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| var req CopyGlobalPricingRequest | |||
| c.ShouldBindJSON(&req) | |||
| // 获取该渠道支持的所有模型(使用 GetAbilitiesByChannelId) | |||
| abilities, err := model.GetAbilitiesByChannelId(channelId) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| imported := 0 | |||
| for _, ability := range abilities { | |||
| // 检查是否已存在 | |||
| existing, _ := model.GetChannelPricing(ability.Model, channelId) | |||
| if existing != nil && !req.Overwrite { | |||
| continue | |||
| } | |||
| // 获取全局扩展比率 | |||
| globalCacheRatio, hasCacheRatio := ratio_setting.GetCacheRatio(ability.Model) | |||
| if !hasCacheRatio { | |||
| globalCacheRatio = 0 | |||
| } | |||
| globalCacheCreationRatio, hasCacheCreationRatio := ratio_setting.GetCreateCacheRatio(ability.Model) | |||
| if !hasCacheCreationRatio { | |||
| globalCacheCreationRatio = 0 | |||
| } | |||
| globalImageRatio, hasImageRatio := ratio_setting.GetImageRatio(ability.Model) | |||
| if !hasImageRatio { | |||
| globalImageRatio = 0 | |||
| } | |||
| globalAudioRatio, hasAudioRatio := ratio_setting.GetAudioRatioV2(ability.Model) | |||
| if !hasAudioRatio { | |||
| globalAudioRatio = 0 | |||
| } | |||
| globalAudioCompletionRatio, hasAudioCompRatio := ratio_setting.GetAudioCompletionRatioV2(ability.Model) | |||
| if !hasAudioCompRatio { | |||
| globalAudioCompletionRatio = 0 | |||
| } | |||
| // 确定定价类型 | |||
| var quotaType int | |||
| var ratio, completionRatio, price float64 | |||
| // 优先检查是否有按次计费的价格 | |||
| modelPrice, hasPrice := ratio_setting.GetModelPrice(ability.Model, false) | |||
| if hasPrice { | |||
| quotaType = model.QuotaTypeByCall | |||
| price = modelPrice | |||
| } else { | |||
| // 使用按量计费 | |||
| quotaType = model.QuotaTypeByTokens | |||
| modelRatio, hasRatio, _ := ratio_setting.GetModelRatio(ability.Model) | |||
| if hasRatio { | |||
| ratio = modelRatio | |||
| } | |||
| completionRatio = ratio_setting.GetCompletionRatio(ability.Model) | |||
| } | |||
| if existing != nil { | |||
| existing.ApplyFields(quotaType, ratio, completionRatio, price, "", | |||
| globalCacheRatio, globalCacheCreationRatio, globalImageRatio, globalAudioRatio, globalAudioCompletionRatio) | |||
| if err := existing.Update(); err == nil { | |||
| imported++ | |||
| } | |||
| } else { | |||
| cp := &model.ChannelPricing{ | |||
| ModelName: ability.Model, | |||
| ChannelId: channelId, | |||
| } | |||
| cp.ApplyFields(quotaType, ratio, completionRatio, price, "", | |||
| globalCacheRatio, globalCacheCreationRatio, globalImageRatio, globalAudioRatio, globalAudioCompletionRatio) | |||
| if err := cp.Insert(); err == nil { | |||
| imported++ | |||
| } | |||
| } | |||
| } | |||
| common.SysLog(fmt.Sprintf("[ChannelPricing] copyGlobalPricing: channel=%d imported=%d/%d", channelId, imported, len(abilities))) | |||
| common.ApiSuccess(c, gin.H{ | |||
| "total": len(abilities), | |||
| "imported": imported, | |||
| }) | |||
| } | |||
| // ChannelPricingWithTags 带标签详情的渠道定价响应 | |||
| type ChannelPricingWithTags struct { | |||
| *model.ChannelPricing | |||
| Tags []*model.PricingTag `json:"tags"` | |||
| } | |||
| // GetChannelPricingByModelWithChannelInfo 获取指定模型的渠道定价(带渠道信息,所有用户可访问) | |||
| func GetChannelPricingByModelWithChannelInfo(c *gin.Context) { | |||
| // 使用通配符路由时,参数包含前导斜杠,需要去除 | |||
| modelName := c.Param("name") | |||
| if modelName == "" { | |||
| common.ApiErrorMsg(c, "model name is required") | |||
| return | |||
| } | |||
| // 去除前导斜杠(路由是 /channel-pricing/model/*name,name 会是 "/deepseek-ai/xxx") | |||
| modelName = strings.TrimPrefix(modelName, "/") | |||
| list, err := model.GetChannelPricingByModelWithChannelInfo(modelName) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| loggedIn, userId, userGroup, groupRatio := resolveCurrentUser(c) | |||
| if loggedIn { | |||
| for _, item := range list { | |||
| userChannelRatio := model.GetUserChannelRatio(userId, modelName, item.ChannelId) | |||
| item.UserRatio = groupRatio * userChannelRatio | |||
| } | |||
| } | |||
| c.JSON(200, gin.H{ | |||
| "success": true, | |||
| "message": "", | |||
| "data": list, | |||
| "logged_in": loggedIn, | |||
| "group": userGroup, | |||
| "group_ratio": groupRatio, | |||
| }) | |||
| } | |||
| // GetChannelPricingWithTags 获取渠道定价(带标签详情) | |||
| func GetChannelPricingWithTags(c *gin.Context) { | |||
| page, _ := strconv.Atoi(c.DefaultQuery("p", "1")) | |||
| pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "10")) | |||
| offset := (page - 1) * pageSize | |||
| list, total, err := model.GetAllChannelPricing(offset, pageSize) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| // 获取所有标签 | |||
| tags, _ := model.GetAllPricingTags() | |||
| tagMap := make(map[int]*model.PricingTag) | |||
| for _, tag := range tags { | |||
| tagMap[tag.Id] = tag | |||
| } | |||
| // 为每个定价填充标签详情 | |||
| result := make([]*ChannelPricingWithTags, 0, len(list)) | |||
| for _, cp := range list { | |||
| item := &ChannelPricingWithTags{ | |||
| ChannelPricing: cp, | |||
| Tags: model.ParseTagIds(cp.TagIds, tagMap), | |||
| } | |||
| result = append(result, item) | |||
| } | |||
| common.ApiSuccess(c, gin.H{ | |||
| "page": page, | |||
| "page_size": pageSize, | |||
| "total": total, | |||
| "items": result, | |||
| }) | |||
| } | |||
| // SetDefaultChannelRequest 设置默认通道请求 | |||
| type SetDefaultChannelRequest struct { | |||
| ModelName string `json:"model_name" binding:"required"` | |||
| ChannelId int `json:"channel_id" binding:"required"` | |||
| } | |||
| // SetDefaultChannel 设置指定模型的默认通道 | |||
| func SetDefaultChannel(c *gin.Context) { | |||
| var req SetDefaultChannelRequest | |||
| if err := c.ShouldBindJSON(&req); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| // 验证渠道定价记录存在 | |||
| existing, err := model.GetChannelPricing(req.ModelName, req.ChannelId) | |||
| if err != nil || existing == nil { | |||
| common.ApiErrorMsg(c, "channel pricing not found for this model and channel") | |||
| return | |||
| } | |||
| if err := model.SetDefaultChannel(req.ModelName, req.ChannelId); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.SysLog(fmt.Sprintf("[ChannelPricing] set default: model=%s channel=%d", req.ModelName, req.ChannelId)) | |||
| common.ApiSuccess(c, nil) | |||
| } | |||
| // ClearDefaultChannel 清除指定模型的默认通道 | |||
| func ClearDefaultChannel(c *gin.Context) { | |||
| modelName := c.Param("name") | |||
| modelName = strings.TrimPrefix(modelName, "/") | |||
| if modelName == "" { | |||
| common.ApiErrorMsg(c, "model name is required") | |||
| return | |||
| } | |||
| if err := model.ClearDefaultChannel(modelName); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.SysLog(fmt.Sprintf("[ChannelPricing] cleared default: model=%s", modelName)) | |||
| common.ApiSuccess(c, nil) | |||
| } | |||
| @@ -0,0 +1,203 @@ | |||
| package controller | |||
| import ( | |||
| "errors" | |||
| "net/http" | |||
| "strconv" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/model" | |||
| user_migration "github.com/QuantumNous/new-api/service/user_migration" | |||
| "github.com/gin-gonic/gin" | |||
| ) | |||
| func ListMigrationUsers(c *gin.Context) { | |||
| page, _ := strconv.Atoi(c.DefaultQuery("page", "0")) | |||
| pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "50")) | |||
| keyword := c.Query("keyword") | |||
| if pageSize <= 0 || pageSize > 200 { | |||
| pageSize = 50 | |||
| } | |||
| users, total, err := model.QueryLocalUsersForMigration(page, pageSize, keyword) | |||
| if err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": "database error"}) | |||
| return | |||
| } | |||
| snapshots := make([]user_migration.RemoteUserSnapshot, 0, len(users)) | |||
| for _, user := range users { | |||
| bindings, _ := model.GetUserOAuthBindingsByUserId(user.Id) | |||
| snapshots = append(snapshots, buildRemoteUserSnapshot(user, bindings)) | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{"success": true, "data": snapshots, "total": total}) | |||
| } | |||
| func QueryMigrationUsers(c *gin.Context) { | |||
| var req user_migration.QueryRemoteUsersRequest | |||
| if err := c.ShouldBindJSON(&req); err != nil { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "invalid request body"}) | |||
| return | |||
| } | |||
| if req.SelectionMode != user_migration.SelectionModeExplicitIDs { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "invalid selection_mode"}) | |||
| return | |||
| } | |||
| if req.PageSize <= 0 || req.PageSize > 200 { | |||
| req.PageSize = 50 | |||
| } | |||
| indexed := make(map[int]struct{}, len(req.SourceUserIDs)) | |||
| orderedIDs := make([]int, 0, len(req.SourceUserIDs)) | |||
| for _, id := range req.SourceUserIDs { | |||
| if id <= 0 { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "invalid source_user_ids"}) | |||
| return | |||
| } | |||
| if _, ok := indexed[id]; ok { | |||
| continue | |||
| } | |||
| indexed[id] = struct{}{} | |||
| orderedIDs = append(orderedIDs, id) | |||
| } | |||
| summary := &user_migration.SelectionSummary{ | |||
| Requested: len(orderedIDs), | |||
| Excluded: make([]user_migration.SelectionExcluded, 0), | |||
| } | |||
| snapshots := make([]user_migration.RemoteUserSnapshot, 0, len(orderedIDs)) | |||
| for _, userID := range orderedIDs { | |||
| user, err := model.GetUserById(userID, true) | |||
| if err != nil || user == nil { | |||
| summary.Excluded = append(summary.Excluded, user_migration.SelectionExcluded{UserID: userID, Reason: "not_found"}) | |||
| continue | |||
| } | |||
| if user.Role == common.RoleRootUser { | |||
| summary.Excluded = append(summary.Excluded, user_migration.SelectionExcluded{UserID: userID, Reason: "root_user_not_migratable"}) | |||
| continue | |||
| } | |||
| if user.Source == common.UserSourceSynced { | |||
| summary.Excluded = append(summary.Excluded, user_migration.SelectionExcluded{UserID: userID, Reason: "already_synced"}) | |||
| continue | |||
| } | |||
| bindings, _ := model.GetUserOAuthBindingsByUserId(userID) | |||
| snapshots = append(snapshots, buildRemoteUserSnapshot(user, bindings)) | |||
| } | |||
| summary.Matched = len(snapshots) | |||
| start := req.Page * req.PageSize | |||
| if start > len(snapshots) { | |||
| start = len(snapshots) | |||
| } | |||
| end := start + req.PageSize | |||
| if end > len(snapshots) { | |||
| end = len(snapshots) | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{ | |||
| "success": true, | |||
| "data": snapshots[start:end], | |||
| "total": len(snapshots), | |||
| "selection_summary": summary, | |||
| }) | |||
| } | |||
| func GetMigrationUser(c *gin.Context) { | |||
| userId, err := strconv.Atoi(c.Param("id")) | |||
| if err != nil { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "invalid user id"}) | |||
| return | |||
| } | |||
| user, err := model.GetUserById(userId, true) | |||
| if err != nil || user == nil { | |||
| c.JSON(http.StatusNotFound, gin.H{"success": false, "error": "user not found"}) | |||
| return | |||
| } | |||
| if user.Role == common.RoleRootUser { | |||
| c.JSON(http.StatusNotFound, gin.H{"success": false, "error": "user not found"}) | |||
| return | |||
| } | |||
| bindings, _ := model.GetUserOAuthBindingsByUserId(userId) | |||
| c.JSON(http.StatusOK, gin.H{"success": true, "data": buildRemoteUserSnapshot(user, bindings)}) | |||
| } | |||
| func ConvertMigrationUserToSynced(c *gin.Context) { | |||
| userId, err := strconv.Atoi(c.Param("id")) | |||
| if err != nil { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "invalid user id"}) | |||
| return | |||
| } | |||
| var req user_migration.ConvertRemoteUserRequest | |||
| if err := c.ShouldBindJSON(&req); err != nil || req.RemoteUserId == 0 { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "invalid request body"}) | |||
| return | |||
| } | |||
| if err := model.ConvertUserToSynced(userId, req.RemoteUserId, req.SyncedQuota); err != nil { | |||
| if errors.Is(err, model.ErrConvertRootUserToSynced) || | |||
| errors.Is(err, model.ErrSyncedUserRemoteUserIDImmutable) { | |||
| c.JSON(http.StatusConflict, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{"success": true}) | |||
| } | |||
| func CheckSyncedCopy(c *gin.Context) { | |||
| cnUserId, err := strconv.Atoi(c.Query("cn_user_id")) | |||
| if err != nil { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "invalid cn_user_id"}) | |||
| return | |||
| } | |||
| hasCopy, err := model.HasOVSyncedCopyByRemoteId(cnUserId) | |||
| if err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{"success": true, "has_copy": hasCopy}) | |||
| } | |||
| func buildRemoteUserSnapshot(user *model.User, bindings []*model.UserOAuthBinding) user_migration.RemoteUserSnapshot { | |||
| snapshot := user_migration.RemoteUserSnapshot{ | |||
| Id: user.Id, | |||
| Username: user.Username, | |||
| Password: user.Password, | |||
| Email: user.Email, | |||
| DisplayName: user.DisplayName, | |||
| Status: user.Status, | |||
| Role: user.Role, | |||
| Group: user.Group, | |||
| Quota: user.Quota, | |||
| AffCode: user.AffCode, | |||
| CreatedAt: user.CreatedAt, | |||
| Setting: user.Setting, | |||
| GitHubId: user.GitHubId, | |||
| DiscordId: user.DiscordId, | |||
| OidcId: user.OidcId, | |||
| WeChatId: user.WeChatId, | |||
| TelegramId: user.TelegramId, | |||
| LinuxDOId: user.LinuxDOId, | |||
| Source: user.Source, | |||
| RemoteUserId: user.RemoteUserId, | |||
| SyncedQuota: user.SyncedQuota, | |||
| } | |||
| for _, binding := range bindings { | |||
| provider, err := model.GetCustomOAuthProviderById(binding.ProviderId) | |||
| if err != nil { | |||
| continue | |||
| } | |||
| snapshot.OAuthBindings = append(snapshot.OAuthBindings, user_migration.RemoteOAuthBinding{ | |||
| ProviderSlug: provider.Slug, | |||
| ProviderUserId: binding.ProviderUserId, | |||
| }) | |||
| } | |||
| return snapshot | |||
| } | |||
| @@ -0,0 +1,491 @@ | |||
| package controller | |||
| import ( | |||
| "encoding/json" | |||
| "net/http" | |||
| "net/http/httptest" | |||
| "strings" | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/model" | |||
| user_migration "github.com/QuantumNous/new-api/service/user_migration" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| func setupInternalMigrationDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, err := db.DB() | |||
| require.NoError(t, err) | |||
| sqlDB.SetMaxOpenConns(1) | |||
| require.NoError(t, db.AutoMigrate(&model.User{}, &model.CustomOAuthProvider{}, &model.UserOAuthBinding{})) | |||
| return db | |||
| } | |||
| func TestListMigrationUsers_ReturnsLocalOnly(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.GET("/api/internal/migration/users", ListMigrationUsers) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000001, | |||
| Username: "ov-1", | |||
| Password: "hash", | |||
| Email: "u1@example.com", | |||
| AffCode: "A1", | |||
| Source: common.UserSourceLocal, | |||
| Quota: 99, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000002, | |||
| Username: "ov-2", | |||
| Password: "hash", | |||
| AffCode: "A2", | |||
| Source: common.UserSourceSynced, | |||
| RemoteUserId: 1, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000003, | |||
| Username: "root", | |||
| Password: "hash", | |||
| AffCode: "A3", | |||
| Source: common.UserSourceLocal, | |||
| Role: common.RoleRootUser, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodGet, "/api/internal/migration/users?page=0&page_size=20", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var body map[string]any | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &body)) | |||
| require.True(t, body["success"].(bool)) | |||
| require.EqualValues(t, 1, body["total"]) | |||
| } | |||
| func TestListMigrationUsers_FiltersByKeyword(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.GET("/api/internal/migration/users", ListMigrationUsers) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000011, | |||
| Username: "alice-local", | |||
| Password: "hash", | |||
| Email: "alice@example.com", | |||
| AffCode: "A11", | |||
| Source: common.UserSourceLocal, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000012, | |||
| Username: "bob-local", | |||
| Password: "hash", | |||
| Email: "bob@example.com", | |||
| AffCode: "A12", | |||
| Source: common.UserSourceLocal, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodGet, "/api/internal/migration/users?page=0&page_size=20&keyword=alice", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var body struct { | |||
| Success bool `json:"success"` | |||
| Data []user_migration.RemoteUserSnapshot `json:"data"` | |||
| Total int `json:"total"` | |||
| } | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &body)) | |||
| require.True(t, body.Success) | |||
| require.Equal(t, 1, body.Total) | |||
| require.Len(t, body.Data, 1) | |||
| require.Equal(t, 10000011, body.Data[0].Id) | |||
| } | |||
| func TestQueryMigrationUsers_OnlyReturnsSelectedLocalUsers(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.POST("/api/internal/migration/users/query", QueryMigrationUsers) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{Id: 10000001, Username: "u1", Password: "hash", AffCode: "A1", Source: common.UserSourceLocal}).Error) | |||
| require.NoError(t, db.Create(&model.User{Id: 10000002, Username: "u2", Password: "hash", AffCode: "A2", Source: common.UserSourceLocal, Role: common.RoleRootUser}).Error) | |||
| require.NoError(t, db.Create(&model.User{Id: 10000003, Username: "u3", Password: "hash", AffCode: "A3", Source: common.UserSourceSynced}).Error) | |||
| req := httptest.NewRequest(http.MethodPost, "/api/internal/migration/users/query", strings.NewReader(`{"page":0,"page_size":50,"selection_mode":"explicit_ids","source_user_ids":[10000001,10000002,10000003,10000004]}`)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var body struct { | |||
| Success bool `json:"success"` | |||
| Data []user_migration.RemoteUserSnapshot `json:"data"` | |||
| Total int `json:"total"` | |||
| SelectionSummary user_migration.SelectionSummary `json:"selection_summary"` | |||
| } | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &body)) | |||
| require.True(t, body.Success) | |||
| require.Len(t, body.Data, 1) | |||
| require.Equal(t, 10000001, body.Data[0].Id) | |||
| require.Equal(t, 1, body.Total) | |||
| require.Equal(t, 4, body.SelectionSummary.Requested) | |||
| require.Equal(t, 1, body.SelectionSummary.Matched) | |||
| require.Len(t, body.SelectionSummary.Excluded, 3) | |||
| } | |||
| func TestQueryMigrationUsers_InvalidSelectionModeRejected(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.POST("/api/internal/migration/users/query", QueryMigrationUsers) | |||
| req := httptest.NewRequest(http.MethodPost, "/api/internal/migration/users/query", strings.NewReader(`{"page":0,"page_size":50,"selection_mode":"all","source_user_ids":[1]}`)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusBadRequest, w.Code) | |||
| } | |||
| func TestGetMigrationUser_RootExcluded(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.GET("/api/internal/migration/users/:id", GetMigrationUser) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000011, | |||
| Username: "root", | |||
| Password: "hash", | |||
| AffCode: "ROOT1", | |||
| Source: common.UserSourceLocal, | |||
| Role: common.RoleRootUser, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodGet, "/api/internal/migration/users/10000011", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusNotFound, w.Code) | |||
| } | |||
| func TestGetMigrationUser_SyncedUserStillReadableForVerify(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.GET("/api/internal/migration/users/:id", GetMigrationUser) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000021, | |||
| Username: "synced-user", | |||
| Password: "hash", | |||
| AffCode: "SYNC1", | |||
| Source: common.UserSourceSynced, | |||
| RemoteUserId: 88, | |||
| SyncedQuota: 123, | |||
| Role: common.RoleCommonUser, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodGet, "/api/internal/migration/users/10000021", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var body map[string]any | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &body)) | |||
| require.True(t, body["success"].(bool)) | |||
| } | |||
| func TestGetMigrationUser_InvalidID(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.GET("/api/internal/migration/users/:id", GetMigrationUser) | |||
| req := httptest.NewRequest(http.MethodGet, "/api/internal/migration/users/not-a-number", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusBadRequest, w.Code) | |||
| } | |||
| func TestConvertMigrationUserToSynced_Idempotent(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.POST("/api/internal/migration/users/:id/convert-to-synced", ConvertMigrationUserToSynced) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000002, | |||
| Username: "ov-2", | |||
| Password: "hash", | |||
| AffCode: "A2", | |||
| Source: common.UserSourceLocal, | |||
| Quota: 66, | |||
| }).Error) | |||
| body := `{"remote_user_id":2002,"synced_quota":66}` | |||
| for i := 0; i < 2; i++ { | |||
| req := httptest.NewRequest(http.MethodPost, | |||
| "/api/internal/migration/users/10000002/convert-to-synced", | |||
| strings.NewReader(body)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| } | |||
| } | |||
| func TestConvertMigrationUserToSynced_ConflictRemoteId(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.POST("/api/internal/migration/users/:id/convert-to-synced", ConvertMigrationUserToSynced) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000003, | |||
| Username: "ov-3", | |||
| Password: "hash", | |||
| AffCode: "A3", | |||
| Source: common.UserSourceSynced, | |||
| RemoteUserId: 5000, | |||
| }).Error) | |||
| body := `{"remote_user_id":9999,"synced_quota":100}` | |||
| req := httptest.NewRequest(http.MethodPost, | |||
| "/api/internal/migration/users/10000003/convert-to-synced", | |||
| strings.NewReader(body)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusConflict, w.Code) | |||
| } | |||
| func TestConvertMigrationUserToSynced_ConflictUsesSentinelErrors(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.POST("/api/internal/migration/users/:id/convert-to-synced", ConvertMigrationUserToSynced) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000013, | |||
| Username: "ov-13", | |||
| Password: "hash", | |||
| AffCode: "A13", | |||
| Source: common.UserSourceSynced, | |||
| RemoteUserId: 5001, | |||
| }).Error) | |||
| body := `{"remote_user_id":9999,"synced_quota":100}` | |||
| req := httptest.NewRequest(http.MethodPost, | |||
| "/api/internal/migration/users/10000013/convert-to-synced", | |||
| strings.NewReader(body)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusConflict, w.Code) | |||
| } | |||
| func TestConvertMigrationUserToSynced_RejectsRootUser(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.POST("/api/internal/migration/users/:id/convert-to-synced", ConvertMigrationUserToSynced) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000012, | |||
| Username: "root", | |||
| Password: "hash", | |||
| AffCode: "ROOT2", | |||
| Source: common.UserSourceLocal, | |||
| Role: common.RoleRootUser, | |||
| }).Error) | |||
| body := `{"remote_user_id":9999,"synced_quota":100}` | |||
| req := httptest.NewRequest(http.MethodPost, | |||
| "/api/internal/migration/users/10000012/convert-to-synced", | |||
| strings.NewReader(body)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusConflict, w.Code) | |||
| } | |||
| func TestConvertMigrationUserToSynced_InvalidRemoteUserID(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.POST("/api/internal/migration/users/:id/convert-to-synced", ConvertMigrationUserToSynced) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000014, | |||
| Username: "ov-14", | |||
| Password: "hash", | |||
| AffCode: "A14", | |||
| Source: common.UserSourceLocal, | |||
| }).Error) | |||
| body := `{"remote_user_id":0,"synced_quota":100}` | |||
| req := httptest.NewRequest(http.MethodPost, | |||
| "/api/internal/migration/users/10000014/convert-to-synced", | |||
| strings.NewReader(body)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusBadRequest, w.Code) | |||
| } | |||
| func TestConvertMigrationUserToSynced_PersistsSyncedFields(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.POST("/api/internal/migration/users/:id/convert-to-synced", ConvertMigrationUserToSynced) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000015, | |||
| Username: "ov-15", | |||
| Password: "hash", | |||
| AffCode: "A15", | |||
| Source: common.UserSourceLocal, | |||
| Quota: 88, | |||
| }).Error) | |||
| body := `{"remote_user_id":2015,"synced_quota":166}` | |||
| req := httptest.NewRequest(http.MethodPost, | |||
| "/api/internal/migration/users/10000015/convert-to-synced", | |||
| strings.NewReader(body)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var saved model.User | |||
| require.NoError(t, db.First(&saved, 10000015).Error) | |||
| require.Equal(t, common.UserSourceSynced, saved.Source) | |||
| require.Equal(t, 2015, saved.RemoteUserId) | |||
| require.Equal(t, 166, saved.SyncedQuota) | |||
| require.Greater(t, saved.LastSyncAt, int64(0)) | |||
| } | |||
| func TestCheckSyncedCopy(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testSyncAuthMiddleware()) | |||
| router.GET("/api/internal/migration/users/check-synced-copy", CheckSyncedCopy) | |||
| db := setupInternalMigrationDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 10000010, | |||
| Username: "synced", | |||
| Password: "hash", | |||
| AffCode: "S1", | |||
| Source: common.UserSourceSynced, | |||
| RemoteUserId: 456, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodGet, | |||
| "/api/internal/migration/users/check-synced-copy?cn_user_id=456", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var body map[string]any | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &body)) | |||
| require.True(t, body["has_copy"].(bool)) | |||
| } | |||
| @@ -1,62 +0,0 @@ | |||
| package controller | |||
| import ( | |||
| "net/http" | |||
| "github.com/QuantumNous/new-api/model" | |||
| "github.com/gin-gonic/gin" | |||
| ) | |||
| func GetModelChannels(c *gin.Context) { | |||
| modelName := c.Query("model") | |||
| if modelName == "" { | |||
| c.JSON(http.StatusBadRequest, gin.H{ | |||
| "success": false, | |||
| "message": "model parameter is required", | |||
| }) | |||
| return | |||
| } | |||
| userId := c.GetInt("id") | |||
| if userId == 0 { | |||
| c.JSON(http.StatusUnauthorized, gin.H{ | |||
| "success": false, | |||
| "message": "unauthorized", | |||
| }) | |||
| return | |||
| } | |||
| userCache, err := model.GetUserCache(userId) | |||
| if err != nil || userCache == nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{ | |||
| "success": false, | |||
| "message": "failed to get user info", | |||
| }) | |||
| return | |||
| } | |||
| userGroup := userCache.Group | |||
| if userGroup == "" { | |||
| c.JSON(http.StatusInternalServerError, gin.H{ | |||
| "success": false, | |||
| "message": "failed to get user group", | |||
| }) | |||
| return | |||
| } | |||
| channels, defaultChannelId, err := model.GetModelChannelsForGroup(modelName, userGroup) | |||
| if err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{ | |||
| "success": false, | |||
| "message": "failed to query channels", | |||
| }) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{ | |||
| "success": true, | |||
| "data": gin.H{ | |||
| "channels": channels, | |||
| "default_channel_id": defaultChannelId, | |||
| }, | |||
| }) | |||
| } | |||
| @@ -115,14 +115,14 @@ func GetUserPricing(c *gin.Context) { | |||
| return | |||
| } | |||
| loggedIn, userId, userGroup, groupRatio := resolveCurrentUser(c) | |||
| loggedIn, _, userGroup, groupRatio := resolveCurrentUser(c) | |||
| if !loggedIn { | |||
| respondOriginalPrice(c, pricingData) | |||
| return | |||
| } | |||
| bestUserChannelRatio := model.GetBestUserChannelRatio(userId, modelName) | |||
| totalRatio := groupRatio * bestUserChannelRatio | |||
| totalRatio := groupRatio | |||
| savingsPercent := int(math.Round((1 - totalRatio) * 100)) | |||
| result := gin.H{ | |||
| @@ -131,7 +131,6 @@ func GetUserPricing(c *gin.Context) { | |||
| "quota_type": pricingData.QuotaType, | |||
| "group": userGroup, | |||
| "group_ratio": groupRatio, | |||
| "user_channel_ratio": bestUserChannelRatio, | |||
| "savings_percent": savingsPercent, | |||
| "logged_in": true, | |||
| } | |||
| @@ -177,15 +176,14 @@ func formatDiscount(ratio float64) string { | |||
| if ratio <= 0 { | |||
| return "免费" | |||
| } | |||
| discount := int(math.Round(ratio * 10)) | |||
| if discount >= 10 { | |||
| rawDiscount := math.Round(ratio*100) / 10 | |||
| if rawDiscount >= 10 { | |||
| return "" | |||
| } | |||
| remainder := int(math.Round(ratio*100)) % 10 | |||
| if remainder == 0 { | |||
| return fmt.Sprintf("%d折", discount) | |||
| if rawDiscount == math.Trunc(rawDiscount) { | |||
| return fmt.Sprintf("%.0f折", rawDiscount) | |||
| } | |||
| return fmt.Sprintf("%.1f折", ratio*10) | |||
| return fmt.Sprintf("%.1f折", rawDiscount) | |||
| } | |||
| func ResetModelRatio(c *gin.Context) { | |||
| @@ -1,131 +0,0 @@ | |||
| package controller | |||
| import ( | |||
| "strconv" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/model" | |||
| "github.com/gin-gonic/gin" | |||
| ) | |||
| // GetAllPricingTags 获取所有定价标签 | |||
| func GetAllPricingTags(c *gin.Context) { | |||
| list, err := model.GetAllPricingTags() | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.ApiSuccess(c, list) | |||
| } | |||
| // CreatePricingTagRequest 创建定价标签请求 | |||
| type CreatePricingTagRequest struct { | |||
| Name string `json:"name" binding:"required"` | |||
| Color string `json:"color"` | |||
| Description string `json:"description"` | |||
| SortOrder int `json:"sort_order"` | |||
| } | |||
| // CreatePricingTag 创建定价标签 | |||
| func CreatePricingTag(c *gin.Context) { | |||
| var req CreatePricingTagRequest | |||
| if err := c.ShouldBindJSON(&req); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| // 检查名称是否重复 | |||
| existing, _ := model.GetPricingTagByName(req.Name) | |||
| if existing != nil { | |||
| common.ApiErrorMsg(c, "tag name already exists") | |||
| return | |||
| } | |||
| pt := &model.PricingTag{ | |||
| Name: req.Name, | |||
| Color: req.Color, | |||
| Description: req.Description, | |||
| SortOrder: req.SortOrder, | |||
| } | |||
| if pt.Color == "" { | |||
| pt.Color = "#1890ff" | |||
| } | |||
| if err := pt.Insert(); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.ApiSuccess(c, pt) | |||
| } | |||
| // UpdatePricingTagRequest 更新定价标签请求 | |||
| type UpdatePricingTagRequest struct { | |||
| Name string `json:"name"` | |||
| Color string `json:"color"` | |||
| Description string `json:"description"` | |||
| SortOrder int `json:"sort_order"` | |||
| } | |||
| // UpdatePricingTag 更新定价标签 | |||
| func UpdatePricingTag(c *gin.Context) { | |||
| idStr := c.Param("id") | |||
| id, err := strconv.Atoi(idStr) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| var req UpdatePricingTagRequest | |||
| if err := c.ShouldBindJSON(&req); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| pt, err := model.GetPricingTagById(id) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| // 检查名称是否与其他标签重复 | |||
| if req.Name != "" && req.Name != pt.Name { | |||
| existing, _ := model.GetPricingTagByName(req.Name) | |||
| if existing != nil && existing.Id != id { | |||
| common.ApiErrorMsg(c, "tag name already exists") | |||
| return | |||
| } | |||
| pt.Name = req.Name | |||
| } | |||
| if req.Color != "" { | |||
| pt.Color = req.Color | |||
| } | |||
| pt.Description = req.Description | |||
| pt.SortOrder = req.SortOrder | |||
| if err := pt.Update(); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.ApiSuccess(c, pt) | |||
| } | |||
| // DeletePricingTag 删除定价标签 | |||
| func DeletePricingTag(c *gin.Context) { | |||
| idStr := c.Param("id") | |||
| id, err := strconv.Atoi(idStr) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| pt := &model.PricingTag{Id: id} | |||
| if err := pt.Delete(); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| common.ApiSuccess(c, nil) | |||
| } | |||
| @@ -29,7 +29,7 @@ func setupPricingTestDB(t *testing.T) *gorm.DB { | |||
| common.UsingSQLite = true | |||
| common.RedisEnabled = false | |||
| require.NoError(t, db.AutoMigrate(&model.UserChannelRatio{}, &model.User{})) | |||
| require.NoError(t, db.AutoMigrate(&model.User{})) | |||
| t.Cleanup(func() { | |||
| model.DB = origDB | |||
| @@ -226,94 +226,7 @@ func TestGetUserPricing_LoggedIn_GroupDiscount(t *testing.T) { | |||
| assert.Equal(t, "8折", resp["discount"]) | |||
| } | |||
| // TestGetUserPricing_LoggedIn_UserChannelRatio 已登录,有用户渠道倍率 | |||
| func TestGetUserPricing_LoggedIn_UserChannelRatio(t *testing.T) { | |||
| db := setupPricingTestDB(t) | |||
| setPricingCache([]model.Pricing{ | |||
| { | |||
| ModelName: "gpt-4o", | |||
| QuotaType: 0, | |||
| ModelRatio: 15, | |||
| CompletionRatio: 4, | |||
| EnableGroup: []string{"default"}, | |||
| }, | |||
| }) | |||
| // 创建用户(default 分组,倍率为1) | |||
| user := &model.User{Id: 300, Group: "default", Username: "specialuser", Status: 1} | |||
| require.NoError(t, db.Create(user).Error) | |||
| // 插入用户渠道倍率 | |||
| ucr := &model.UserChannelRatio{ | |||
| UserId: 300, | |||
| ModelName: "gpt-4o", | |||
| ChannelId: 1, | |||
| Ratio: 0.9, | |||
| } | |||
| require.NoError(t, ucr.Insert()) | |||
| router := setupAuthRouter(300) | |||
| w := httptest.NewRecorder() | |||
| req, _ := http.NewRequest("GET", "/api/pricing/user/gpt-4o", nil) | |||
| router.ServeHTTP(w, req) | |||
| var resp map[string]interface{} | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| assert.True(t, resp["success"].(bool)) | |||
| assert.Equal(t, true, resp["logged_in"]) | |||
| // group_ratio=1 * user_channel_ratio=0.9 = 0.9 | |||
| assert.Equal(t, float64(0.9), resp["user_channel_ratio"]) | |||
| assert.Equal(t, float64(10), resp["savings_percent"]) | |||
| // 输入用户价: 30 * 0.9 = 27 | |||
| assert.Equal(t, float64(27), resp["user_input"]) | |||
| } | |||
| // TestGetUserPricing_LoggedIn_BothDiscounts 分组倍率 + 用户渠道倍率叠加 | |||
| func TestGetUserPricing_LoggedIn_BothDiscounts(t *testing.T) { | |||
| db := setupPricingTestDB(t) | |||
| setPricingCache([]model.Pricing{ | |||
| { | |||
| ModelName: "gpt-4o", | |||
| QuotaType: 0, | |||
| ModelRatio: 15, | |||
| CompletionRatio: 4, | |||
| EnableGroup: []string{"default", "vip"}, | |||
| }, | |||
| }) | |||
| user := &model.User{Id: 400, Group: "vip", Username: "bothdiscount", Status: 1} | |||
| require.NoError(t, db.Create(user).Error) | |||
| // VIP 分组倍率 0.8 | |||
| withGroupRatio(t, `{"default":1,"vip":0.8}`) | |||
| // 用户渠道倍率 0.9 | |||
| ucr := &model.UserChannelRatio{ | |||
| UserId: 400, | |||
| ModelName: "gpt-4o", | |||
| ChannelId: 1, | |||
| Ratio: 0.9, | |||
| } | |||
| require.NoError(t, ucr.Insert()) | |||
| router := setupAuthRouter(400) | |||
| w := httptest.NewRecorder() | |||
| req, _ := http.NewRequest("GET", "/api/pricing/user/gpt-4o", nil) | |||
| router.ServeHTTP(w, req) | |||
| var resp map[string]interface{} | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| assert.True(t, resp["success"].(bool)) | |||
| assert.Equal(t, true, resp["logged_in"]) | |||
| // total = 0.8 * 0.9 = 0.72, savings = 28% | |||
| assert.Equal(t, float64(28), resp["savings_percent"]) | |||
| // 输入: 30 * 0.72 = 21.6 | |||
| assert.Equal(t, float64(21.6), resp["user_input"]) | |||
| // 输出: 120 * 0.72 = 86.4 | |||
| assert.Equal(t, float64(86.4), resp["user_output"]) | |||
| } | |||
| // TestGetUserPricing_PerCall_WithDiscount 按次计费 + 折扣 | |||
| func TestGetUserPricing_PerCall_WithDiscount(t *testing.T) { | |||
| @@ -368,43 +281,3 @@ func TestFormatDiscount(t *testing.T) { | |||
| } | |||
| } | |||
| // TestGetUserPricing_MultipleUserChannelRatios 多个渠道倍率取最低值 | |||
| func TestGetUserPricing_MultipleUserChannelRatios(t *testing.T) { | |||
| db := setupPricingTestDB(t) | |||
| setPricingCache([]model.Pricing{ | |||
| { | |||
| ModelName: "gpt-4o", | |||
| QuotaType: 0, | |||
| ModelRatio: 15, | |||
| CompletionRatio: 4, | |||
| EnableGroup: []string{"default"}, | |||
| }, | |||
| }) | |||
| user := &model.User{Id: 600, Group: "default", Username: "multichannel", Status: 1} | |||
| require.NoError(t, db.Create(user).Error) | |||
| // 多个渠道倍率,取最低值 0.7 | |||
| for i, ratio := range []float64{0.9, 0.7, 0.8} { | |||
| ucr := &model.UserChannelRatio{ | |||
| UserId: 600, | |||
| ModelName: "gpt-4o", | |||
| ChannelId: i + 1, | |||
| Ratio: ratio, | |||
| } | |||
| require.NoError(t, ucr.Insert()) | |||
| } | |||
| router := setupAuthRouter(600) | |||
| w := httptest.NewRecorder() | |||
| req, _ := http.NewRequest("GET", "/api/pricing/user/gpt-4o", nil) | |||
| router.ServeHTTP(w, req) | |||
| var resp map[string]interface{} | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| assert.True(t, resp["success"].(bool)) | |||
| // 应取最低倍率 0.7 | |||
| assert.Equal(t, float64(0.7), resp["user_channel_ratio"]) | |||
| assert.Equal(t, float64(30), resp["savings_percent"]) | |||
| } | |||
| @@ -6,10 +6,12 @@ import ( | |||
| "io" | |||
| "log" | |||
| "net/http" | |||
| "strconv" | |||
| "strings" | |||
| "time" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/common/metrics" | |||
| "github.com/QuantumNous/new-api/constant" | |||
| "github.com/QuantumNous/new-api/dto" | |||
| "github.com/QuantumNous/new-api/logger" | |||
| @@ -87,6 +89,15 @@ func Relay(c *gin.Context, relayFormat types.RelayFormat) { | |||
| defer func() { | |||
| if newAPIError != nil { | |||
| logger.LogError(c, fmt.Sprintf("relay error: %s", newAPIError.Error())) | |||
| // Prometheus: 错误计数 | |||
| if metrics.IsEnabled() { | |||
| channelId := strconv.Itoa(c.GetInt("channel_id")) | |||
| modelName := c.GetString("original_model") | |||
| errorType, errorCode := metrics.ClassifyError(newAPIError.StatusCode, newAPIError.Error()) | |||
| metrics.GetMetrics().RequestErrorsTotal.WithLabelValues(channelId, modelName, errorType, errorCode).Inc() | |||
| } | |||
| newAPIError.SetMessage(common.MessageWithRequestId(newAPIError.Error(), requestId)) | |||
| switch relayFormat { | |||
| case types.RelayFormatOpenAIRealtime: | |||
| @@ -193,8 +204,6 @@ func Relay(c *gin.Context, relayFormat types.RelayFormat) { | |||
| break | |||
| } | |||
| // 在渠道选择后更新价格数据以使用渠道定价 | |||
| helper.UpdatePriceDataForChannelPricing(c, relayInfo, channel.Id) | |||
| addUsedChannel(c, channel.Id) | |||
| bodyStorage, bodyErr := common.GetBodyStorage(c) | |||
| @@ -231,6 +240,12 @@ func Relay(c *gin.Context, relayFormat types.RelayFormat) { | |||
| if !shouldRetry(c, newAPIError, common.RetryTimes-retryParam.GetRetry()) { | |||
| break | |||
| } | |||
| // Prometheus: 重试计数 | |||
| if metrics.IsEnabled() { | |||
| channelIdStr := strconv.Itoa(channel.Id) | |||
| metrics.GetMetrics().RequestRetriesTotal.WithLabelValues(channelIdStr, relayInfo.OriginModelName).Inc() | |||
| } | |||
| } | |||
| useChannel := c.GetStringSlice("use_channel") | |||
| @@ -328,9 +343,6 @@ func shouldRetry(c *gin.Context, openaiErr *types.NewAPIError, retryTimes int) b | |||
| if retryTimes <= 0 { | |||
| return false | |||
| } | |||
| if _, ok := c.Get("specific_channel_id"); ok { | |||
| return false | |||
| } | |||
| code := openaiErr.StatusCode | |||
| if code >= 200 && code < 300 { | |||
| return false | |||
| @@ -614,9 +626,6 @@ func shouldRetryTaskRelay(c *gin.Context, channelId int, taskErr *dto.TaskError, | |||
| if retryTimes <= 0 { | |||
| return false | |||
| } | |||
| if _, ok := c.Get("specific_channel_id"); ok { | |||
| return false | |||
| } | |||
| if taskErr.StatusCode == http.StatusTooManyRequests { | |||
| return true | |||
| } | |||
| @@ -0,0 +1,152 @@ | |||
| package controller | |||
| import ( | |||
| "errors" | |||
| "net/http" | |||
| "net/http/httptest" | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/dto" | |||
| "github.com/QuantumNous/new-api/types" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/stretchr/testify/assert" | |||
| ) | |||
| // --------------------------------------------------------------------------- | |||
| // shouldRetry — 渠道选择删除后的验证测试 | |||
| // --------------------------------------------------------------------------- | |||
| func newTestContext(t *testing.T) *gin.Context { | |||
| t.Helper() | |||
| gin.SetMode(gin.TestMode) | |||
| w := httptest.NewRecorder() | |||
| c, _ := gin.CreateTestContext(w) | |||
| c.Request, _ = http.NewRequest("POST", "/v1/chat/completions", nil) | |||
| return c | |||
| } | |||
| func TestShouldRetry_NilError_ReturnsFalse(t *testing.T) { | |||
| c := newTestContext(t) | |||
| assert.False(t, shouldRetry(c, nil, 3)) | |||
| } | |||
| func TestShouldRetry_5xxErrorWithRetryLeft_ReturnsTrue(t *testing.T) { | |||
| // 渠道选择删除后,所有请求都能正常重试 | |||
| // 之前设置了 specific_channel_id 时 shouldRetry 会返回 false | |||
| // 现在即使不设置任何 channel 相关 context,5xx 也应该重试 | |||
| c := newTestContext(t) | |||
| apiErr := types.NewErrorWithStatusCode( | |||
| errors.New("upstream error"), | |||
| types.ErrorCodeBadResponseStatusCode, | |||
| http.StatusInternalServerError, | |||
| ) | |||
| assert.True(t, shouldRetry(c, apiErr, 3)) | |||
| } | |||
| func TestShouldRetry_5xxErrorZeroRetryLeft_ReturnsFalse(t *testing.T) { | |||
| c := newTestContext(t) | |||
| apiErr := types.NewErrorWithStatusCode( | |||
| errors.New("upstream error"), | |||
| types.ErrorCodeBadResponseStatusCode, | |||
| http.StatusInternalServerError, | |||
| ) | |||
| assert.False(t, shouldRetry(c, apiErr, 0)) | |||
| } | |||
| func TestShouldRetry_SkipRetryError_ReturnsFalse(t *testing.T) { | |||
| c := newTestContext(t) | |||
| apiErr := types.NewError( | |||
| errors.New("skip retry"), | |||
| types.ErrorCodeBadResponseStatusCode, | |||
| types.ErrOptionWithSkipRetry(), | |||
| ) | |||
| assert.False(t, shouldRetry(c, apiErr, 3)) | |||
| } | |||
| func TestShouldRetry_ChannelError_ReturnsTrue(t *testing.T) { | |||
| // channel: 前缀的错误码会触发立即重试 | |||
| c := newTestContext(t) | |||
| apiErr := types.NewError( | |||
| errors.New("channel error"), | |||
| "channel:test", | |||
| ) | |||
| // 即使 retryTimes=0,channel error 也返回 true | |||
| assert.True(t, shouldRetry(c, apiErr, 0)) | |||
| } | |||
| func TestShouldRetry_2xxStatus_ReturnsFalse(t *testing.T) { | |||
| c := newTestContext(t) | |||
| apiErr := types.NewErrorWithStatusCode( | |||
| errors.New("ok"), | |||
| types.ErrorCodeBadResponseStatusCode, | |||
| http.StatusOK, | |||
| ) | |||
| assert.False(t, shouldRetry(c, apiErr, 3)) | |||
| } | |||
| func TestShouldRetry_429Status_ReturnsTrue(t *testing.T) { | |||
| c := newTestContext(t) | |||
| apiErr := types.NewErrorWithStatusCode( | |||
| errors.New("rate limited"), | |||
| types.ErrorCodeBadResponseStatusCode, | |||
| http.StatusTooManyRequests, | |||
| ) | |||
| assert.True(t, shouldRetry(c, apiErr, 3)) | |||
| } | |||
| // --------------------------------------------------------------------------- | |||
| // shouldRetryTaskRelay — 渠道选择删除后的验证测试 | |||
| // --------------------------------------------------------------------------- | |||
| func TestShouldRetryTaskRelay_NilError_ReturnsFalse(t *testing.T) { | |||
| c := newTestContext(t) | |||
| assert.False(t, shouldRetryTaskRelay(c, 1, nil, 3)) | |||
| } | |||
| func TestShouldRetryTaskRelay_5xxWithRetryLeft_ReturnsTrue(t *testing.T) { | |||
| c := newTestContext(t) | |||
| taskErr := &dto.TaskError{ | |||
| Error: errors.New("server error"), | |||
| StatusCode: http.StatusInternalServerError, | |||
| } | |||
| assert.True(t, shouldRetryTaskRelay(c, 1, taskErr, 3)) | |||
| } | |||
| func TestShouldRetryTaskRelay_5xxZeroRetryLeft_ReturnsFalse(t *testing.T) { | |||
| c := newTestContext(t) | |||
| taskErr := &dto.TaskError{ | |||
| Error: errors.New("server error"), | |||
| StatusCode: http.StatusInternalServerError, | |||
| } | |||
| assert.False(t, shouldRetryTaskRelay(c, 1, taskErr, 0)) | |||
| } | |||
| func TestShouldRetryTaskRelay_429_ReturnsTrue(t *testing.T) { | |||
| c := newTestContext(t) | |||
| taskErr := &dto.TaskError{ | |||
| Error: errors.New("rate limited"), | |||
| StatusCode: http.StatusTooManyRequests, | |||
| } | |||
| assert.True(t, shouldRetryTaskRelay(c, 1, taskErr, 1)) | |||
| } | |||
| func TestShouldRetryTaskRelay_400_ReturnsFalse(t *testing.T) { | |||
| c := newTestContext(t) | |||
| taskErr := &dto.TaskError{ | |||
| Error: errors.New("bad request"), | |||
| StatusCode: http.StatusBadRequest, | |||
| } | |||
| assert.False(t, shouldRetryTaskRelay(c, 1, taskErr, 3)) | |||
| } | |||
| func TestShouldRetryTaskRelay_LocalError_ReturnsFalse(t *testing.T) { | |||
| // LocalError + 非 5xx 状态码(如 403)时,LocalError 检查生效返回 false | |||
| // 注意:5xx 分支在 LocalError 检查之前,所以 localError+5xx 仍会重试 | |||
| c := newTestContext(t) | |||
| taskErr := &dto.TaskError{ | |||
| Error: errors.New("local error"), | |||
| StatusCode: http.StatusForbidden, | |||
| LocalError: true, | |||
| } | |||
| assert.False(t, shouldRetryTaskRelay(c, 1, taskErr, 3)) | |||
| } | |||
| @@ -562,6 +562,16 @@ func UpdateUser(c *gin.Context) { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| if originUser.IsSyncedUser() { | |||
| // 对同步用户而言,界面展示和用户感知的“余额”是 synced_quota, | |||
| // 不能用本地 quota 列做比较,否则在 quota / synced_quota 漂移后会误判。 | |||
| if originUser.SyncedQuota != updatedUser.Quota { | |||
| common.ApiErrorI18n(c, i18n.MsgSyncedUserQuotaCannotModify) | |||
| return | |||
| } | |||
| // 提交其他字段编辑时,保留本地 quota 原值,避免把展示用 synced_quota 回写到 quota 列。 | |||
| updatedUser.Quota = originUser.Quota | |||
| } | |||
| myRole := c.GetInt("role") | |||
| if myRole <= originUser.Role && myRole != common.RoleRootUser { | |||
| common.ApiErrorI18n(c, i18n.MsgUserNoPermissionHigherLevel) | |||
| @@ -1025,6 +1035,10 @@ func TopUp(c *gin.Context) { | |||
| } | |||
| quota, err := model.Redeem(req.Key, id) | |||
| if err != nil { | |||
| if errors.Is(err, model.ErrSyncedUserRedeemDenied) { | |||
| common.ApiErrorI18n(c, i18n.MsgRedemptionSyncedUserDenied) | |||
| return | |||
| } | |||
| if errors.Is(err, model.ErrRedeemFailed) { | |||
| common.ApiErrorI18n(c, i18n.MsgRedeemFailed) | |||
| return | |||
| @@ -1,116 +0,0 @@ | |||
| package controller | |||
| import ( | |||
| "net/http" | |||
| "strconv" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/model" | |||
| "github.com/gin-gonic/gin" | |||
| ) | |||
| func GetUserChannelRatios(c *gin.Context) { | |||
| userIdStr := c.Param("user_id") | |||
| userId, err := strconv.Atoi(userIdStr) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| ratios, err := model.GetUserChannelRatiosByUserId(userId) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{ | |||
| "success": true, | |||
| "data": ratios, | |||
| }) | |||
| } | |||
| type createUserChannelRatioRequest struct { | |||
| UserId int `json:"user_id"` | |||
| ModelName string `json:"model_name"` | |||
| ChannelId int `json:"channel_id"` | |||
| Ratio float64 `json:"ratio"` | |||
| } | |||
| func CreateUserChannelRatio(c *gin.Context) { | |||
| var req createUserChannelRatioRequest | |||
| if err := c.ShouldBindJSON(&req); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| if req.Ratio <= 0 { | |||
| common.ApiErrorMsg(c, "ratio must be positive") | |||
| return | |||
| } | |||
| ucr := &model.UserChannelRatio{ | |||
| UserId: req.UserId, | |||
| ModelName: req.ModelName, | |||
| ChannelId: req.ChannelId, | |||
| Ratio: req.Ratio, | |||
| } | |||
| if err := ucr.Insert(); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{ | |||
| "success": true, | |||
| "data": ucr, | |||
| }) | |||
| } | |||
| func UpdateUserChannelRatio(c *gin.Context) { | |||
| idStr := c.Param("id") | |||
| id, err := strconv.Atoi(idStr) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| var req struct { | |||
| Ratio float64 `json:"ratio"` | |||
| } | |||
| if err := c.ShouldBindJSON(&req); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| if req.Ratio <= 0 { | |||
| common.ApiErrorMsg(c, "ratio must be positive") | |||
| return | |||
| } | |||
| ucr := &model.UserChannelRatio{ | |||
| Id: id, | |||
| Ratio: req.Ratio, | |||
| } | |||
| if err := ucr.Update(); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{ | |||
| "success": true, | |||
| "data": ucr, | |||
| }) | |||
| } | |||
| func DeleteUserChannelRatio(c *gin.Context) { | |||
| idStr := c.Param("id") | |||
| id, err := strconv.Atoi(idStr) | |||
| if err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| if err := model.DeleteUserChannelRatioById(id); err != nil { | |||
| common.ApiError(c, err) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{ | |||
| "success": true, | |||
| }) | |||
| } | |||
| @@ -0,0 +1,376 @@ | |||
| package controller | |||
| import ( | |||
| "errors" | |||
| "net/http" | |||
| "strconv" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/model" | |||
| user_migration "github.com/QuantumNous/new-api/service/user_migration" | |||
| "github.com/QuantumNous/new-api/setting/system_setting" | |||
| "github.com/gin-gonic/gin" | |||
| "gorm.io/gorm" | |||
| ) | |||
| type migrationVerifier interface { | |||
| VerifyItem(item *model.UserMigrationItem) (*user_migration.VerifyResult, error) | |||
| } | |||
| type migrationCandidateClient interface { | |||
| ListUsers(page, pageSize int, keyword string) (*user_migration.ListRemoteUsersResponse, error) | |||
| } | |||
| var newMigrationVerifier = func() migrationVerifier { | |||
| return newMigrationService() | |||
| } | |||
| var newMigrationCandidateClient = func() migrationCandidateClient { | |||
| settings := system_setting.GetRegionSyncSettings() | |||
| endpoint := settings.MasterEndpoint | |||
| if settings.IsMaster && len(settings.SlaveEndpoints) > 0 { | |||
| endpoint = settings.SlaveEndpoints[0] | |||
| } | |||
| return user_migration.NewClient(endpoint, settings.SyncApiKey) | |||
| } | |||
| func CreateUserMigrationBatch(c *gin.Context) { | |||
| var req struct { | |||
| Name string `json:"name" binding:"required"` | |||
| SourceRegion string `json:"source_region" binding:"required"` | |||
| TargetRegion string `json:"target_region" binding:"required"` | |||
| SelectionMode string `json:"selection_mode"` | |||
| SourceUserIDs []int `json:"source_user_ids"` | |||
| } | |||
| if err := c.ShouldBindJSON(&req); err != nil { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| operatorID, _ := c.Get("id") | |||
| if req.SelectionMode == "" { | |||
| req.SelectionMode = user_migration.SelectionModeAll | |||
| } | |||
| if req.SelectionMode != user_migration.SelectionModeAll && req.SelectionMode != user_migration.SelectionModeExplicitIDs { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "invalid selection_mode"}) | |||
| return | |||
| } | |||
| payload := user_migration.SelectionPayload{} | |||
| if req.SelectionMode == user_migration.SelectionModeExplicitIDs { | |||
| if len(req.SourceUserIDs) == 0 { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "source_user_ids is required"}) | |||
| return | |||
| } | |||
| seen := make(map[int]struct{}, len(req.SourceUserIDs)) | |||
| payload.SourceUserIDs = make([]int, 0, len(req.SourceUserIDs)) | |||
| for _, id := range req.SourceUserIDs { | |||
| if id <= 0 { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "invalid source_user_ids"}) | |||
| return | |||
| } | |||
| if _, ok := seen[id]; ok { | |||
| continue | |||
| } | |||
| seen[id] = struct{}{} | |||
| payload.SourceUserIDs = append(payload.SourceUserIDs, id) | |||
| } | |||
| if len(payload.SourceUserIDs) > 1000 { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": "too many source_user_ids"}) | |||
| return | |||
| } | |||
| conflicts, err := model.FindActiveMigrationUserConflicts(payload.SourceUserIDs) | |||
| if err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| if len(conflicts) > 0 { | |||
| c.JSON(http.StatusConflict, gin.H{ | |||
| "success": false, | |||
| "error": "some selected users already exist in active migration batches", | |||
| "conflicts": conflicts, | |||
| }) | |||
| return | |||
| } | |||
| } | |||
| payloadJSON, _ := common.Marshal(payload) | |||
| batch := &model.UserMigrationBatch{ | |||
| Name: req.Name, | |||
| SourceRegion: req.SourceRegion, | |||
| TargetRegion: req.TargetRegion, | |||
| SelectionMode: req.SelectionMode, | |||
| SelectionPayload: string(payloadJSON), | |||
| RequestedUserCount: len(payload.SourceUserIDs), | |||
| OperatorId: operatorID.(int), | |||
| } | |||
| if err := model.CreateUserMigrationBatch(batch); err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{"success": true, "data": batch}) | |||
| } | |||
| func ListMigrationCandidateUsers(c *gin.Context) { | |||
| page, _ := strconv.Atoi(c.DefaultQuery("page", "0")) | |||
| pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "50")) | |||
| keyword := c.Query("keyword") | |||
| if pageSize <= 0 || pageSize > 200 { | |||
| pageSize = 50 | |||
| } | |||
| resp, err := newMigrationCandidateClient().ListUsers(page, pageSize, keyword) | |||
| if err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{ | |||
| "success": true, | |||
| "data": resp.Data, | |||
| "total": resp.Total, | |||
| }) | |||
| } | |||
| func ListUserMigrationBatches(c *gin.Context) { | |||
| var batches []model.UserMigrationBatch | |||
| if err := model.DB.Order("id desc").Find(&batches).Error; err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{"success": true, "data": batches}) | |||
| } | |||
| func GetUserMigrationBatch(c *gin.Context) { | |||
| batchID, _ := strconv.Atoi(c.Param("id")) | |||
| var batch model.UserMigrationBatch | |||
| if err := model.DB.First(&batch, batchID).Error; err != nil { | |||
| c.JSON(http.StatusNotFound, gin.H{"success": false, "error": "batch not found"}) | |||
| return | |||
| } | |||
| page, _ := strconv.Atoi(c.DefaultQuery("page", "0")) | |||
| pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "50")) | |||
| if pageSize <= 0 { | |||
| pageSize = 50 | |||
| } | |||
| if pageSize > 200 { | |||
| pageSize = 200 | |||
| } | |||
| var items []model.UserMigrationItem | |||
| var total int64 | |||
| model.DB.Model(&model.UserMigrationItem{}).Where("batch_id = ?", batchID).Count(&total) | |||
| model.DB.Where("batch_id = ?", batchID).Order("id asc").Offset(page * pageSize).Limit(pageSize).Find(&items) | |||
| c.JSON(http.StatusOK, gin.H{ | |||
| "success": true, | |||
| "data": batch, | |||
| "items": items, | |||
| "total": total, | |||
| }) | |||
| } | |||
| func ScanUserMigrationBatch(c *gin.Context) { | |||
| batchID, _ := strconv.Atoi(c.Param("id")) | |||
| locked, err := model.TryLockBatchForScan(batchID) | |||
| if err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| if !locked { | |||
| c.JSON(http.StatusConflict, gin.H{"success": false, "error": "batch is already being scanned or not in a scannable state"}) | |||
| return | |||
| } | |||
| svc := newMigrationService() | |||
| go func() { | |||
| if err := svc.ScanBatch(batchID); err != nil { | |||
| model.DB.Model(&model.UserMigrationBatch{}).Where("id = ?", batchID). | |||
| Update("status", model.UserMigrationBatchStatusFailed) | |||
| } | |||
| }() | |||
| c.JSON(http.StatusAccepted, gin.H{"success": true, "message": "scan started"}) | |||
| } | |||
| func ResolveUserMigrationItem(c *gin.Context) { | |||
| itemID, _ := strconv.Atoi(c.Param("id")) | |||
| var item model.UserMigrationItem | |||
| if err := model.DB.Select("id", "batch_id", "conflict_flags").First(&item, itemID).Error; err != nil { | |||
| c.JSON(http.StatusNotFound, gin.H{"success": false, "error": "item not found"}) | |||
| return | |||
| } | |||
| var req struct { | |||
| ResolutionStrategy string `json:"resolution_strategy" binding:"required"` | |||
| TargetUserId int `json:"target_user_id"` | |||
| } | |||
| if err := c.ShouldBindJSON(&req); err != nil { | |||
| c.JSON(http.StatusBadRequest, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| validStrategies := map[string]bool{ | |||
| model.UserMigrationStrategyCreateNew: true, | |||
| model.UserMigrationStrategyMergeExisting: true, | |||
| model.UserMigrationStrategySkip: true, | |||
| } | |||
| if !validStrategies[req.ResolutionStrategy] { | |||
| c.JSON(http.StatusBadRequest, gin.H{ | |||
| "success": false, | |||
| "error": "invalid resolution_strategy: must be create_new, merge_into_existing, or skip", | |||
| }) | |||
| return | |||
| } | |||
| if req.ResolutionStrategy == model.UserMigrationStrategyMergeExisting && req.TargetUserId == 0 { | |||
| c.JSON(http.StatusBadRequest, gin.H{ | |||
| "success": false, | |||
| "error": "merge_into_existing requires target_user_id", | |||
| }) | |||
| return | |||
| } | |||
| if req.ResolutionStrategy == model.UserMigrationStrategyMergeExisting && | |||
| hasMigrationConflictFlag(item.ConflictFlags, "cn_already_synced_to_ov") { | |||
| c.JSON(http.StatusConflict, gin.H{ | |||
| "success": false, | |||
| "error": "merge_into_existing is not allowed when target CN user already has a synced OV copy", | |||
| }) | |||
| return | |||
| } | |||
| updates := map[string]any{ | |||
| "resolution_strategy": req.ResolutionStrategy, | |||
| "status": model.UserMigrationItemStatusReady, | |||
| } | |||
| if req.TargetUserId != 0 { | |||
| updates["target_user_id"] = req.TargetUserId | |||
| } | |||
| if err := model.DB.Model(&model.UserMigrationItem{}).Where("id = ?", itemID).Updates(updates).Error; err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| if err := model.RefreshUserMigrationBatchStats(item.BatchId); err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{"success": true}) | |||
| } | |||
| func hasMigrationConflictFlag(flagsJSON string, want string) bool { | |||
| if flagsJSON == "" || want == "" { | |||
| return false | |||
| } | |||
| var flags []string | |||
| if err := common.UnmarshalJsonStr(flagsJSON, &flags); err != nil { | |||
| return false | |||
| } | |||
| for _, flag := range flags { | |||
| if flag == want { | |||
| return true | |||
| } | |||
| } | |||
| return false | |||
| } | |||
| func ExecuteUserMigrationBatch(c *gin.Context) { | |||
| batchID, _ := strconv.Atoi(c.Param("id")) | |||
| locked, err := model.TryLockBatchForExecution(batchID) | |||
| if err != nil { | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| if !locked { | |||
| c.JSON(http.StatusConflict, gin.H{"success": false, "error": "batch is already running or not in an executable state"}) | |||
| return | |||
| } | |||
| svc := newMigrationService() | |||
| go func() { | |||
| _ = svc.ExecuteBatch(batchID) | |||
| }() | |||
| c.JSON(http.StatusAccepted, gin.H{"success": true, "message": "execution started"}) | |||
| } | |||
| func RetryUserMigrationBatch(c *gin.Context) { | |||
| batchID, _ := strconv.Atoi(c.Param("id")) | |||
| model.DB.Model(&model.UserMigrationItem{}). | |||
| Where("batch_id = ? AND status = ?", batchID, model.UserMigrationItemStatusFailed). | |||
| Updates(map[string]any{"status": model.UserMigrationItemStatusReady, "error_message": ""}) | |||
| locked, err := model.TryLockBatchForExecution(batchID) | |||
| if err != nil || !locked { | |||
| c.JSON(http.StatusConflict, gin.H{"success": false, "error": "batch cannot be retried now"}) | |||
| return | |||
| } | |||
| svc := newMigrationService() | |||
| go func() { | |||
| _ = svc.ExecuteBatch(batchID) | |||
| }() | |||
| c.JSON(http.StatusAccepted, gin.H{"success": true, "message": "retry started"}) | |||
| } | |||
| func CancelUserMigrationBatch(c *gin.Context) { | |||
| batchID, _ := strconv.Atoi(c.Param("id")) | |||
| cancelled, err := model.CancelUserMigrationBatch(batchID) | |||
| if err != nil { | |||
| if errors.Is(err, gorm.ErrRecordNotFound) { | |||
| c.JSON(http.StatusNotFound, gin.H{"success": false, "error": "batch not found"}) | |||
| return | |||
| } | |||
| c.JSON(http.StatusInternalServerError, gin.H{"success": false, "error": err.Error()}) | |||
| return | |||
| } | |||
| if !cancelled { | |||
| c.JSON(http.StatusConflict, gin.H{ | |||
| "success": false, | |||
| "error": "batch cannot be cancelled in current status", | |||
| }) | |||
| return | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{"success": true, "message": "batch cancelled"}) | |||
| } | |||
| func VerifyUserMigrationBatch(c *gin.Context) { | |||
| batchID, _ := strconv.Atoi(c.Param("id")) | |||
| var items []model.UserMigrationItem | |||
| model.DB.Where("batch_id = ? AND status = ?", batchID, model.UserMigrationItemStatusMigrated).Find(&items) | |||
| svc := newMigrationVerifier() | |||
| results := make([]map[string]any, 0, len(items)) | |||
| for _, item := range items { | |||
| result, err := svc.VerifyItem(&item) | |||
| entry := map[string]any{ | |||
| "source_user_id": item.SourceUserId, | |||
| "target_user_id": item.TargetUserId, | |||
| } | |||
| if err != nil { | |||
| entry["error"] = err.Error() | |||
| } else { | |||
| entry["result"] = result | |||
| } | |||
| results = append(results, entry) | |||
| } | |||
| c.JSON(http.StatusOK, gin.H{"success": true, "data": results}) | |||
| } | |||
| func newMigrationService() *user_migration.Service { | |||
| settings := system_setting.GetRegionSyncSettings() | |||
| endpoint := settings.MasterEndpoint | |||
| if settings.IsMaster && len(settings.SlaveEndpoints) > 0 { | |||
| endpoint = settings.SlaveEndpoints[0] | |||
| } | |||
| client := user_migration.NewClient(endpoint, settings.SyncApiKey) | |||
| return user_migration.NewService(client) | |||
| } | |||
| @@ -0,0 +1,839 @@ | |||
| package controller | |||
| import ( | |||
| "bytes" | |||
| "encoding/json" | |||
| "errors" | |||
| "net/http" | |||
| "net/http/httptest" | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/model" | |||
| user_migration "github.com/QuantumNous/new-api/service/user_migration" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| func setupUserMigrationControllerDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, err := db.DB() | |||
| require.NoError(t, err) | |||
| sqlDB.SetMaxOpenConns(1) | |||
| require.NoError(t, db.AutoMigrate( | |||
| &model.User{}, | |||
| &model.CustomOAuthProvider{}, | |||
| &model.UserOAuthBinding{}, | |||
| &model.UserMigrationBatch{}, | |||
| &model.UserMigrationItem{}, | |||
| &model.MigrationQuotaGrant{}, | |||
| )) | |||
| return db | |||
| } | |||
| func testRootAuthMiddleware() gin.HandlerFunc { | |||
| return func(c *gin.Context) { | |||
| c.Set("id", 1) | |||
| c.Set("role", 100) | |||
| c.Next() | |||
| } | |||
| } | |||
| type fakeMigrationCandidateClient struct { | |||
| resp *user_migration.ListRemoteUsersResponse | |||
| err error | |||
| gotKeyword string | |||
| gotPage int | |||
| gotPageSize int | |||
| } | |||
| func (f *fakeMigrationCandidateClient) ListUsers(page, pageSize int, keyword string) (*user_migration.ListRemoteUsersResponse, error) { | |||
| f.gotKeyword = keyword | |||
| f.gotPage = page | |||
| f.gotPageSize = pageSize | |||
| if f.err != nil { | |||
| return nil, f.err | |||
| } | |||
| return f.resp, nil | |||
| } | |||
| type fakeMigrationVerifier struct { | |||
| results map[int]*user_migration.VerifyResult | |||
| errs map[int]error | |||
| } | |||
| func (f *fakeMigrationVerifier) VerifyItem(item *model.UserMigrationItem) (*user_migration.VerifyResult, error) { | |||
| if err, ok := f.errs[item.SourceUserId]; ok { | |||
| return nil, err | |||
| } | |||
| if result, ok := f.results[item.SourceUserId]; ok { | |||
| return result, nil | |||
| } | |||
| return nil, errors.New("unexpected source user") | |||
| } | |||
| func TestCreateUserMigrationBatch_Success(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches", CreateUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| body := map[string]any{ | |||
| "name": "wave-root-1", | |||
| "source_region": "overseas", | |||
| "target_region": "cn", | |||
| } | |||
| bodyBytes, err := json.Marshal(body) | |||
| require.NoError(t, err) | |||
| req := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches", bytes.NewBuffer(bodyBytes)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var resp map[string]any | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| require.True(t, resp["success"].(bool)) | |||
| require.NotNil(t, resp["data"]) | |||
| } | |||
| func TestCreateUserMigrationBatch_ExplicitIDsSuccess(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches", CreateUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| req := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches", bytes.NewBufferString(`{ | |||
| "name":"explicit-batch", | |||
| "source_region":"ov", | |||
| "target_region":"cn", | |||
| "selection_mode":"explicit_ids", | |||
| "source_user_ids":[1001,1002,1002] | |||
| }`)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var batch model.UserMigrationBatch | |||
| require.NoError(t, db.First(&batch, 1).Error) | |||
| require.Equal(t, "explicit_ids", batch.SelectionMode) | |||
| require.Equal(t, 2, batch.RequestedUserCount) | |||
| } | |||
| func TestCreateUserMigrationBatch_ExplicitIDsValidation(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches", CreateUserMigrationBatch) | |||
| cases := []string{ | |||
| `{"name":"b1","source_region":"ov","target_region":"cn","selection_mode":"explicit_ids","source_user_ids":[]}`, | |||
| `{"name":"b2","source_region":"ov","target_region":"cn","selection_mode":"explicit_ids","source_user_ids":[0]}`, | |||
| `{"name":"b3","source_region":"ov","target_region":"cn","selection_mode":"invalid","source_user_ids":[1]}`, | |||
| } | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| for _, body := range cases { | |||
| req := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches", bytes.NewBufferString(body)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusBadRequest, w.Code) | |||
| } | |||
| } | |||
| func TestCreateUserMigrationBatch_ExplicitIDs_RejectsActiveBatchOverlap(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches", CreateUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| existingBatch := &model.UserMigrationBatch{ | |||
| Name: "existing-draft-batch", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| SelectionMode: "explicit_ids", | |||
| SelectionPayload: `{"source_user_ids":[1002]}`, | |||
| Status: model.UserMigrationBatchStatusDraft, | |||
| OperatorId: 1, | |||
| } | |||
| require.NoError(t, db.Create(existingBatch).Error) | |||
| req := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches", bytes.NewBufferString(`{ | |||
| "name":"explicit-batch", | |||
| "source_region":"ov", | |||
| "target_region":"cn", | |||
| "selection_mode":"explicit_ids", | |||
| "source_user_ids":[1001,1002] | |||
| }`)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusConflict, w.Code) | |||
| var resp struct { | |||
| Success bool `json:"success"` | |||
| Error string `json:"error"` | |||
| Conflicts []struct { | |||
| SourceUserID int `json:"source_user_id"` | |||
| BatchID int `json:"batch_id"` | |||
| BatchStatus string `json:"batch_status"` | |||
| } `json:"conflicts"` | |||
| } | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| require.False(t, resp.Success) | |||
| require.Equal(t, "some selected users already exist in active migration batches", resp.Error) | |||
| require.Len(t, resp.Conflicts, 1) | |||
| require.Equal(t, 1002, resp.Conflicts[0].SourceUserID) | |||
| require.Equal(t, existingBatch.Id, resp.Conflicts[0].BatchID) | |||
| require.Equal(t, model.UserMigrationBatchStatusDraft, resp.Conflicts[0].BatchStatus) | |||
| } | |||
| func TestListMigrationCandidateUsers_Success(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| origFactory := newMigrationCandidateClient | |||
| defer func() { | |||
| newMigrationCandidateClient = origFactory | |||
| }() | |||
| newMigrationCandidateClient = func() migrationCandidateClient { | |||
| return &fakeMigrationCandidateClient{ | |||
| resp: &user_migration.ListRemoteUsersResponse{ | |||
| Success: true, | |||
| Data: []user_migration.RemoteUserSnapshot{ | |||
| {Id: 10000001, Username: "ov-a", Email: "a@example.com", Quota: 100}, | |||
| }, | |||
| Total: 1, | |||
| }, | |||
| } | |||
| } | |||
| router.GET("/api/user-migrations/candidate-users", ListMigrationCandidateUsers) | |||
| req := httptest.NewRequest(http.MethodGet, "/api/user-migrations/candidate-users?page=0&page_size=50", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var resp map[string]any | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| require.True(t, resp["success"].(bool)) | |||
| require.EqualValues(t, 1, resp["total"]) | |||
| } | |||
| func TestListMigrationCandidateUsers_UpstreamFailure(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| origFactory := newMigrationCandidateClient | |||
| defer func() { | |||
| newMigrationCandidateClient = origFactory | |||
| }() | |||
| newMigrationCandidateClient = func() migrationCandidateClient { | |||
| return &fakeMigrationCandidateClient{err: errors.New("upstream unavailable")} | |||
| } | |||
| router.GET("/api/user-migrations/candidate-users", ListMigrationCandidateUsers) | |||
| req := httptest.NewRequest(http.MethodGet, "/api/user-migrations/candidate-users?page=0&page_size=50", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusInternalServerError, w.Code) | |||
| } | |||
| func TestListMigrationCandidateUsers_ForwardsKeyword(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| client := &fakeMigrationCandidateClient{ | |||
| resp: &user_migration.ListRemoteUsersResponse{ | |||
| Success: true, | |||
| Data: []user_migration.RemoteUserSnapshot{}, | |||
| Total: 0, | |||
| }, | |||
| } | |||
| origFactory := newMigrationCandidateClient | |||
| defer func() { | |||
| newMigrationCandidateClient = origFactory | |||
| }() | |||
| newMigrationCandidateClient = func() migrationCandidateClient { | |||
| return client | |||
| } | |||
| router.GET("/api/user-migrations/candidate-users", ListMigrationCandidateUsers) | |||
| req := httptest.NewRequest(http.MethodGet, "/api/user-migrations/candidate-users?page=2&page_size=30&keyword=alice", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| require.Equal(t, 2, client.gotPage) | |||
| require.Equal(t, 30, client.gotPageSize) | |||
| require.Equal(t, "alice", client.gotKeyword) | |||
| } | |||
| func TestListUserMigrationBatches_Success(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.GET("/api/user-migrations/batches", ListUserMigrationBatches) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.UserMigrationBatch{ | |||
| Id: 1, | |||
| Name: "batch-1", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&model.UserMigrationBatch{ | |||
| Id: 2, | |||
| Name: "batch-2", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodGet, "/api/user-migrations/batches", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var resp map[string]any | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| require.True(t, resp["success"].(bool)) | |||
| require.Len(t, resp["data"], 2) | |||
| } | |||
| func TestGetUserMigrationBatch_Success(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.GET("/api/user-migrations/batches/:id", GetUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.UserMigrationBatch{ | |||
| Id: 1, | |||
| Name: "detail-batch", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&model.UserMigrationItem{ | |||
| BatchId: 1, | |||
| SourceUserId: 1001, | |||
| SourceUsername: "ov-u1", | |||
| Status: model.UserMigrationItemStatusReady, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodGet, "/api/user-migrations/batches/1?page=0&page_size=50", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var resp map[string]any | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| require.True(t, resp["success"].(bool)) | |||
| require.NotNil(t, resp["data"]) | |||
| require.Len(t, resp["items"], 1) | |||
| require.EqualValues(t, 1, resp["total"]) | |||
| } | |||
| func TestGetUserMigrationBatch_NotFound(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.GET("/api/user-migrations/batches/:id", GetUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| req := httptest.NewRequest(http.MethodGet, "/api/user-migrations/batches/999", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusNotFound, w.Code) | |||
| } | |||
| func TestExecuteUserMigrationBatch_AlreadyRunning(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches/:id/execute", ExecuteUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| batch := &model.UserMigrationBatch{ | |||
| Name: "b", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| Status: model.UserMigrationBatchStatusRunning, | |||
| } | |||
| require.NoError(t, db.Create(batch).Error) | |||
| req := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/execute", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusConflict, w.Code) | |||
| } | |||
| func TestExecuteUserMigrationBatch_InvalidStateRejected(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches/:id/execute", ExecuteUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| for _, status := range []string{model.UserMigrationBatchStatusDraft, model.UserMigrationBatchStatusScanned} { | |||
| require.NoError(t, db.Exec("DELETE FROM user_migration_batches").Error) | |||
| require.NoError(t, db.Create(&model.UserMigrationBatch{ | |||
| Id: 1, | |||
| Name: "invalid-exec", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| Status: status, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/execute", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusConflict, w.Code) | |||
| } | |||
| } | |||
| func TestResolveUserMigrationItem_InvalidStrategy(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/items/:id/resolve", ResolveUserMigrationItem) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.UserMigrationItem{ | |||
| BatchId: 1, | |||
| SourceUserId: 10000001, | |||
| SourceUsername: "u", | |||
| Status: model.UserMigrationItemStatusConflict, | |||
| }).Error) | |||
| req := httptest.NewRequest( | |||
| http.MethodPost, | |||
| "/api/user-migrations/items/1/resolve", | |||
| bytes.NewBufferString(`{"resolution_strategy":"invalid_strategy"}`), | |||
| ) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusBadRequest, w.Code) | |||
| } | |||
| func TestResolveUserMigrationItem_RefreshesBatchStatusToReady(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/items/:id/resolve", ResolveUserMigrationItem) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| batch := &model.UserMigrationBatch{ | |||
| Name: "resolve-ready", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| Status: model.UserMigrationBatchStatusScanned, | |||
| } | |||
| require.NoError(t, model.CreateUserMigrationBatch(batch)) | |||
| require.NoError(t, db.Create(&model.UserMigrationItem{ | |||
| BatchId: batch.Id, | |||
| SourceUserId: 1001, | |||
| SourceUsername: "ov-u1", | |||
| Status: model.UserMigrationItemStatusConflict, | |||
| }).Error) | |||
| req := httptest.NewRequest( | |||
| http.MethodPost, | |||
| "/api/user-migrations/items/1/resolve", | |||
| bytes.NewBufferString(`{"resolution_strategy":"create_new"}`), | |||
| ) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var refreshed model.UserMigrationBatch | |||
| require.NoError(t, db.First(&refreshed, batch.Id).Error) | |||
| require.Equal(t, model.UserMigrationBatchStatusReady, refreshed.Status) | |||
| } | |||
| func TestResolveUserMigrationItem_RejectsMergeWhenCNAlreadyHasSyncedCopy(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/items/:id/resolve", ResolveUserMigrationItem) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.UserMigrationItem{ | |||
| Id: 1, | |||
| BatchId: 1, | |||
| SourceUserId: 1001, | |||
| SourceUsername: "ov-u1", | |||
| Status: model.UserMigrationItemStatusConflict, | |||
| ConflictFlags: `["email","cn_already_synced_to_ov"]`, | |||
| TargetUserId: 24, | |||
| }).Error) | |||
| req := httptest.NewRequest( | |||
| http.MethodPost, | |||
| "/api/user-migrations/items/1/resolve", | |||
| bytes.NewBufferString(`{"resolution_strategy":"merge_into_existing","target_user_id":24}`), | |||
| ) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusConflict, w.Code) | |||
| var item model.UserMigrationItem | |||
| require.NoError(t, db.First(&item, 1).Error) | |||
| require.Equal(t, model.UserMigrationItemStatusConflict, item.Status) | |||
| require.Equal(t, 24, item.TargetUserId) | |||
| } | |||
| func TestRetryUserMigrationBatch_OnlyFailedItemsReset(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches/:id/retry", RetryUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.UserMigrationBatch{ | |||
| Id: 1, | |||
| Name: "retry-batch", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| Status: model.UserMigrationBatchStatusRunning, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&model.UserMigrationItem{ | |||
| Id: 1, | |||
| BatchId: 1, | |||
| SourceUserId: 1001, | |||
| SourceUsername: "failed-user", | |||
| Status: model.UserMigrationItemStatusFailed, | |||
| ErrorMessage: "boom", | |||
| }).Error) | |||
| require.NoError(t, db.Create(&model.UserMigrationItem{ | |||
| Id: 2, | |||
| BatchId: 1, | |||
| SourceUserId: 1002, | |||
| SourceUsername: "migrated-user", | |||
| Status: model.UserMigrationItemStatusMigrated, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&model.UserMigrationItem{ | |||
| Id: 3, | |||
| BatchId: 1, | |||
| SourceUserId: 1003, | |||
| SourceUsername: "skipped-user", | |||
| Status: model.UserMigrationItemStatusSkipped, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/retry", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusConflict, w.Code) | |||
| var failedItem, migratedItem, skippedItem model.UserMigrationItem | |||
| require.NoError(t, db.First(&failedItem, 1).Error) | |||
| require.NoError(t, db.First(&migratedItem, 2).Error) | |||
| require.NoError(t, db.First(&skippedItem, 3).Error) | |||
| require.Equal(t, model.UserMigrationItemStatusReady, failedItem.Status) | |||
| require.Empty(t, failedItem.ErrorMessage) | |||
| require.Equal(t, model.UserMigrationItemStatusMigrated, migratedItem.Status) | |||
| require.Equal(t, model.UserMigrationItemStatusSkipped, skippedItem.Status) | |||
| } | |||
| func TestCancelUserMigrationBatch_SoftDeleteSuccess(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches/:id/cancel", CancelUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&model.UserMigrationBatch{ | |||
| Id: 1, | |||
| Name: "cancel-me", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| SelectionMode: "explicit_ids", | |||
| SelectionPayload: `{"source_user_ids":[11,12]}`, | |||
| OperatorId: 1, | |||
| Status: model.UserMigrationBatchStatusScanned, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/cancel", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var batch model.UserMigrationBatch | |||
| require.NoError(t, db.First(&batch, 1).Error) | |||
| require.Equal(t, model.UserMigrationBatchStatusCancelled, batch.Status) | |||
| } | |||
| func TestCancelUserMigrationBatch_InvalidStateRejected(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches/:id/cancel", CancelUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| for _, status := range []string{ | |||
| model.UserMigrationBatchStatusScanning, | |||
| model.UserMigrationBatchStatusRunning, | |||
| model.UserMigrationBatchStatusCompleted, | |||
| } { | |||
| require.NoError(t, db.Exec("DELETE FROM user_migration_batches").Error) | |||
| require.NoError(t, db.Create(&model.UserMigrationBatch{ | |||
| Id: 1, | |||
| Name: "cannot-cancel", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| Status: status, | |||
| }).Error) | |||
| req := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/cancel", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusConflict, w.Code) | |||
| } | |||
| } | |||
| func TestCreateUserMigrationBatch_ExplicitIDs_AllowsReuseAfterCancel(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches", CreateUserMigrationBatch) | |||
| router.POST("/api/user-migrations/batches/:id/cancel", CancelUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| }() | |||
| existingBatch := &model.UserMigrationBatch{ | |||
| Name: "existing-scanned-batch", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| SelectionMode: "explicit_ids", | |||
| SelectionPayload: `{"source_user_ids":[11,12]}`, | |||
| Status: model.UserMigrationBatchStatusScanned, | |||
| OperatorId: 1, | |||
| } | |||
| require.NoError(t, db.Create(existingBatch).Error) | |||
| cancelReq := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/cancel", nil) | |||
| cancelW := httptest.NewRecorder() | |||
| router.ServeHTTP(cancelW, cancelReq) | |||
| require.Equal(t, http.StatusOK, cancelW.Code) | |||
| createReq := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches", bytes.NewBufferString(`{ | |||
| "name":"recreated-batch", | |||
| "source_region":"ov", | |||
| "target_region":"cn", | |||
| "selection_mode":"explicit_ids", | |||
| "source_user_ids":[11,12] | |||
| }`)) | |||
| createReq.Header.Set("Content-Type", "application/json") | |||
| createW := httptest.NewRecorder() | |||
| router.ServeHTTP(createW, createReq) | |||
| require.Equal(t, http.StatusOK, createW.Code) | |||
| } | |||
| func TestVerifyUserMigrationBatch_AggregatesResult(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.POST("/api/user-migrations/batches/:id/verify", VerifyUserMigrationBatch) | |||
| db := setupUserMigrationControllerDB(t) | |||
| orig := model.DB | |||
| origFactory := newMigrationVerifier | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| newMigrationVerifier = origFactory | |||
| }() | |||
| require.NoError(t, db.Create(&model.UserMigrationItem{ | |||
| Id: 1, | |||
| BatchId: 1, | |||
| SourceUserId: 1001, | |||
| TargetUserId: 1, | |||
| Status: model.UserMigrationItemStatusMigrated, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&model.UserMigrationItem{ | |||
| Id: 2, | |||
| BatchId: 1, | |||
| SourceUserId: 1002, | |||
| TargetUserId: 2, | |||
| Status: model.UserMigrationItemStatusMigrated, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&model.UserMigrationItem{ | |||
| Id: 3, | |||
| BatchId: 1, | |||
| SourceUserId: 1003, | |||
| TargetUserId: 3, | |||
| Status: model.UserMigrationItemStatusMigrated, | |||
| }).Error) | |||
| newMigrationVerifier = func() migrationVerifier { | |||
| return &fakeMigrationVerifier{ | |||
| results: map[int]*user_migration.VerifyResult{ | |||
| 1001: { | |||
| TargetUserExists: true, | |||
| RemoteConverted: true, | |||
| QuotaMatched: true, | |||
| }, | |||
| 1002: { | |||
| TargetUserExists: false, | |||
| RemoteConverted: true, | |||
| QuotaMatched: false, | |||
| }, | |||
| }, | |||
| errs: map[int]error{ | |||
| 1003: errors.New("remote user not found"), | |||
| }, | |||
| } | |||
| } | |||
| req := httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/verify", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var resp struct { | |||
| Success bool `json:"success"` | |||
| Data []map[string]any `json:"data"` | |||
| } | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| require.True(t, resp.Success) | |||
| require.Len(t, resp.Data, 3) | |||
| bySource := make(map[int]map[string]any, len(resp.Data)) | |||
| for _, entry := range resp.Data { | |||
| bySource[int(entry["source_user_id"].(float64))] = entry | |||
| } | |||
| require.NotNil(t, bySource[1001]["result"]) | |||
| require.NotNil(t, bySource[1002]["result"]) | |||
| require.Equal(t, "remote user not found", bySource[1003]["error"]) | |||
| } | |||
| @@ -0,0 +1,146 @@ | |||
| package controller | |||
| import ( | |||
| "bytes" | |||
| "encoding/json" | |||
| "net/http" | |||
| "net/http/httptest" | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/model" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| func setupUserControllerDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, err := db.DB() | |||
| require.NoError(t, err) | |||
| sqlDB.SetMaxOpenConns(1) | |||
| require.NoError(t, db.AutoMigrate(&model.User{})) | |||
| return db | |||
| } | |||
| func TestUpdateUser_SyncedUserAllowsNonQuotaEditsWithoutOverwritingQuota(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.PUT("/api/user/", UpdateUser) | |||
| db := setupUserControllerDB(t) | |||
| orig := model.DB | |||
| origRedisEnabled := common.RedisEnabled | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| common.RedisEnabled = origRedisEnabled | |||
| }() | |||
| common.RedisEnabled = false | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 1001, | |||
| Username: "synced-user", | |||
| Password: "hashed-password", | |||
| DisplayName: "Old Name", | |||
| Role: common.RoleCommonUser, | |||
| Status: common.UserStatusEnabled, | |||
| Group: "default", | |||
| Quota: 0, | |||
| SyncedQuota: 100000, | |||
| Source: common.UserSourceSynced, | |||
| AffCode: "SYNC1", | |||
| }).Error) | |||
| body := map[string]any{ | |||
| "id": 1001, | |||
| "username": "synced-user", | |||
| "display_name": "New Name", | |||
| "password": "", | |||
| "group": "default", | |||
| "quota": 100000, | |||
| } | |||
| bodyBytes, err := json.Marshal(body) | |||
| require.NoError(t, err) | |||
| req := httptest.NewRequest(http.MethodPut, "/api/user/", bytes.NewBuffer(bodyBytes)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var resp map[string]any | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| require.True(t, resp["success"].(bool)) | |||
| var saved model.User | |||
| require.NoError(t, db.First(&saved, 1001).Error) | |||
| require.Equal(t, "New Name", saved.DisplayName) | |||
| require.Equal(t, 0, saved.Quota) | |||
| require.Equal(t, 100000, saved.SyncedQuota) | |||
| } | |||
| func TestUpdateUser_SyncedUserQuotaChangeRejected(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| router := gin.New() | |||
| router.Use(testRootAuthMiddleware()) | |||
| router.PUT("/api/user/", UpdateUser) | |||
| db := setupUserControllerDB(t) | |||
| orig := model.DB | |||
| origRedisEnabled := common.RedisEnabled | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| common.RedisEnabled = origRedisEnabled | |||
| }() | |||
| common.RedisEnabled = false | |||
| require.NoError(t, db.Create(&model.User{ | |||
| Id: 1002, | |||
| Username: "synced-user-2", | |||
| Password: "hashed-password", | |||
| DisplayName: "Synced User 2", | |||
| Role: common.RoleCommonUser, | |||
| Status: common.UserStatusEnabled, | |||
| Group: "default", | |||
| Quota: 0, | |||
| SyncedQuota: 100000, | |||
| Source: common.UserSourceSynced, | |||
| AffCode: "SYNC2", | |||
| }).Error) | |||
| body := map[string]any{ | |||
| "id": 1002, | |||
| "username": "synced-user-2", | |||
| "display_name": "Synced User 2", | |||
| "password": "", | |||
| "group": "default", | |||
| "quota": 100001, | |||
| } | |||
| bodyBytes, err := json.Marshal(body) | |||
| require.NoError(t, err) | |||
| req := httptest.NewRequest(http.MethodPut, "/api/user/", bytes.NewBuffer(bodyBytes)) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| var resp map[string]any | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) | |||
| require.False(t, resp["success"].(bool)) | |||
| require.Equal(t, "user.synced_user_quota_cannot_modify", resp["message"]) | |||
| var saved model.User | |||
| require.NoError(t, db.First(&saved, 1002).Error) | |||
| require.Equal(t, 0, saved.Quota) | |||
| require.Equal(t, 100000, saved.SyncedQuota) | |||
| } | |||
| @@ -0,0 +1,106 @@ | |||
| package controller | |||
| import ( | |||
| "bytes" | |||
| "net/http" | |||
| "net/http/httptest" | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/i18n" | |||
| "github.com/QuantumNous/new-api/model" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/assert" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| func setupUserTopUpTestDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, err := db.DB() | |||
| require.NoError(t, err) | |||
| sqlDB.SetMaxOpenConns(1) | |||
| origDB := model.DB | |||
| origLogDB := model.LOG_DB | |||
| origUsingSQLite := common.UsingSQLite | |||
| origRedisEnabled := common.RedisEnabled | |||
| model.DB = db | |||
| model.LOG_DB = db | |||
| common.UsingSQLite = true | |||
| common.RedisEnabled = false | |||
| require.NoError(t, db.AutoMigrate(&model.User{}, &model.Redemption{}, &model.Log{})) | |||
| t.Cleanup(func() { | |||
| model.DB = origDB | |||
| model.LOG_DB = origLogDB | |||
| common.UsingSQLite = origUsingSQLite | |||
| common.RedisEnabled = origRedisEnabled | |||
| require.NoError(t, sqlDB.Close()) | |||
| }) | |||
| return db | |||
| } | |||
| func setupUserTopUpRouter(userID int) *gin.Engine { | |||
| gin.SetMode(gin.TestMode) | |||
| r := gin.New() | |||
| r.POST("/api/user/topup", func(c *gin.Context) { | |||
| c.Set("id", userID) | |||
| c.Next() | |||
| }, TopUp) | |||
| return r | |||
| } | |||
| func TestTopUp_SyncedUserDenied(t *testing.T) { | |||
| db := setupUserTopUpTestDB(t) | |||
| i18n.Init() | |||
| user := &model.User{ | |||
| Id: 100, | |||
| Username: "synced-topup-test", | |||
| Password: "hashed_password", | |||
| Quota: 100000, | |||
| Source: common.UserSourceSynced, | |||
| } | |||
| require.NoError(t, db.Create(user).Error) | |||
| redemption := &model.Redemption{ | |||
| Id: 1, | |||
| UserId: 1, | |||
| Key: "synced-topup-key", | |||
| Status: common.RedemptionCodeStatusEnabled, | |||
| Quota: 50000, | |||
| CreatedTime: common.GetTimestamp(), | |||
| } | |||
| require.NoError(t, db.Create(redemption).Error) | |||
| router := setupUserTopUpRouter(100) | |||
| w := httptest.NewRecorder() | |||
| reqBody, err := common.Marshal(map[string]string{"key": "synced-topup-key"}) | |||
| require.NoError(t, err) | |||
| req, err := http.NewRequest(http.MethodPost, "/api/user/topup", bytes.NewReader(reqBody)) | |||
| require.NoError(t, err) | |||
| req.Header.Set("Content-Type", "application/json") | |||
| req.Header.Set("Accept-Language", "zh-CN") | |||
| router.ServeHTTP(w, req) | |||
| assert.Equal(t, http.StatusOK, w.Code) | |||
| var resp map[string]interface{} | |||
| require.NoError(t, common.Unmarshal(w.Body.Bytes(), &resp)) | |||
| assert.False(t, resp["success"].(bool)) | |||
| assert.Equal(t, "同步用户不允许使用兑换码", resp["message"]) | |||
| var updatedUser model.User | |||
| require.NoError(t, db.First(&updatedUser, 100).Error) | |||
| assert.Equal(t, 100000, updatedUser.Quota) | |||
| } | |||
| @@ -0,0 +1,295 @@ | |||
| # Token API 参考文档 | |||
| > 本文档以仓库当前实现为准。`controller.GetTokenStatus` 存在,但未在路由中注册;当前可公开调用的 token 相关端点见下文。 | |||
| ## 通用约定 | |||
| ### 基础信息 | |||
| | 项目 | 说明 | | |||
| |------|------| | |||
| | Base URL | `{host}/api` | | |||
| | 认证方式 | 除 `/api/usage/token/` 外,均使用登录态鉴权 | | |||
| | 响应格式 | JSON | | |||
| | 字符编码 | UTF-8 | | |||
| ### 鉴权说明 | |||
| | 场景 | 说明 | | |||
| |------|------| | |||
| | `/api/token/*` | 需要登录态。优先使用 session cookie;若无 session,也可用系统 `access_token` 放在 `Authorization` 头里,但仍必须携带 `New-Api-User` | | |||
| | `/api/usage/token/` | 只需要 `Authorization: Bearer sk-...`,不需要 session 和 `New-Api-User` | | |||
| > session cookie 名由服务端 `SESSION_NAME` 决定,默认是 `session`。 | |||
| > | |||
| > `New-Api-User` 必须与当前登录身份的用户 ID 一致,否则返回: | |||
| > ```json | |||
| > {"success": false, "message": "无权进行此操作,未提供 New-Api-User"} | |||
| > ``` | |||
| > ```json | |||
| > {"success": false, "message": "无权进行此操作,New-Api-User 与登录用户不匹配"} | |||
| > ``` | |||
| ### 统一响应 | |||
| 大多数接口成功时返回: | |||
| ```json | |||
| {"success": true, "message": "", "data": {}} | |||
| ``` | |||
| 业务失败通常返回: | |||
| ```json | |||
| {"success": false, "message": "错误描述"} | |||
| ``` | |||
| > 注意:鉴权失败、限流等中间件可能直接返回 `401/403/429`,不一定是 `200`。 | |||
| ### 分页参数 | |||
| | 参数 | 别名 | 说明 | 默认 | 上限 | | |||
| |------|------|------|------|------| | |||
| | `p` | - | 页码,从 1 开始 | 1 | - | | |||
| | `page_size` | `ps` / `size` | 每页条数 | 10 | 100 | | |||
| ### Token 数据模型 | |||
| > `key` 为原始密钥,不含 `sk-` 前缀;创建接口不回传密钥,列表/详情/搜索会返回该原始值。 | |||
| ```json | |||
| { | |||
| "id": 123, | |||
| "user_id": 44, | |||
| "key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", | |||
| "status": 1, | |||
| "name": "my-token", | |||
| "created_time": 1716163200, | |||
| "accessed_time": 1716249600, | |||
| "expired_time": -1, | |||
| "remain_quota": 500000, | |||
| "unlimited_quota": false, | |||
| "model_limits_enabled": true, | |||
| "model_limits": "gpt-4o,claude-3-5-sonnet", | |||
| "allow_ips": "192.168.1.0/24\n10.0.0.1", | |||
| "used_quota": 120000, | |||
| "group": "default", | |||
| "cross_group_retry": false | |||
| } | |||
| ``` | |||
| | 字段 | 说明 | | |||
| |------|------| | |||
| | `status` | `1` 启用,`2` 禁用,`3` 过期,`4` 耗尽 | | |||
| | `expired_time` | Unix 秒级时间戳,`-1` 表示永不过期 | | |||
| | `remain_quota` | 剩余额度,单位为内部 quota | | |||
| | `model_limits` | 逗号分隔的模型列表 | | |||
| | `allow_ips` | 换行分隔的 IP 白名单,空则不限制 | | |||
| | `group` | 空字符串表示使用用户默认分组 | | |||
| | `cross_group_retry` | 仅 `auto` 分组有意义 | | |||
| ## API 端点 | |||
| ### 1. 获取令牌列表 | |||
| ``` | |||
| GET /api/token/ | |||
| ``` | |||
| #### 请求 | |||
| | 参数 | 说明 | | |||
| |------|------| | |||
| | `p` | 页码 | | |||
| | `page_size` / `ps` / `size` | 每页条数 | | |||
| #### 响应 | |||
| `data` 为分页结构;其中每个 token 的 `key` 是原始密钥,不带 `sk-` 前缀。 | |||
| ### 2. 搜索令牌 | |||
| ``` | |||
| GET /api/token/search | |||
| ``` | |||
| #### 请求 | |||
| | 参数 | 说明 | | |||
| |------|------| | |||
| | `keyword` | 按名称搜索 | | |||
| | `token` | 按密钥搜索,`sk-` 前缀可选 | | |||
| | `p` | 页码 | | |||
| | `page_size` / `ps` / `size` | 每页条数 | | |||
| #### 搜索规则 | |||
| - 不含 `%` 时为精确匹配 | |||
| - 含 `%` 时为模糊匹配 | |||
| - 最多允许 2 个 `%` | |||
| - 不允许连续 `%%` | |||
| - 含 `%` 时,去掉 `%` 后长度必须至少 2 | |||
| - 令牌数量超过上限时,只允许精确搜索 | |||
| ### 3. 获取单个令牌 | |||
| ``` | |||
| GET /api/token/:id | |||
| ``` | |||
| 返回单个 token 对象,`key` 同样是原始密钥。 | |||
| ### 4. 创建令牌 | |||
| ``` | |||
| POST /api/token/ | |||
| ``` | |||
| #### 请求体 | |||
| | 字段 | 说明 | | |||
| |------|------| | |||
| | `name` | 必填,最长 50 字符 | | |||
| | `remain_quota` | 可选,默认 0 | | |||
| | `unlimited_quota` | 可选,默认 false | | |||
| | `expired_time` | 可选,默认 -1 | | |||
| | `model_limits_enabled` | 可选,默认 false | | |||
| | `model_limits` | 可选,逗号分隔字符串 | | |||
| | `allow_ips` | 可选,换行分隔字符串 | | |||
| | `group` | 可选,空字符串表示用户默认分组 | | |||
| | `cross_group_retry` | 可选,默认 false | | |||
| #### 说明 | |||
| - 后端自动生成 `key` | |||
| - 创建成功不返回 `key` | |||
| - 创建成功后通常去列表页复制原始 `key` | |||
| ### 5. 更新令牌 | |||
| ``` | |||
| PUT /api/token/ | |||
| ``` | |||
| #### 请求 | |||
| | 参数 | 说明 | | |||
| |------|------| | |||
| | `status_only=true` | 只更新状态 | | |||
| | 字段 | 说明 | | |||
| |------|------| | |||
| | `id` | 必填 | | |||
| | `name` | 可选 | | |||
| | `status` | 仅 `status_only=true` 时生效 | | |||
| | `remain_quota` | 可选 | | |||
| | `unlimited_quota` | 可选 | | |||
| | `expired_time` | 可选 | | |||
| | `model_limits_enabled` | 可选 | | |||
| | `model_limits` | 可选 | | |||
| | `allow_ips` | 可选 | | |||
| | `group` | 可选 | | |||
| | `cross_group_retry` | 可选 | | |||
| #### 业务规则 | |||
| - 已过期且过期时间已到的 token 不能重新启用 | |||
| - 已耗尽且非无限额度的 token 不能重新启用 | |||
| - 非无限额度时,`remain_quota` 不能为负,也不能超过 `10^9 × QuotaPerUnit` | |||
| - 不带 `status_only=true` 时,`status` 字段会被忽略 | |||
| #### 响应 | |||
| 返回更新后的 token 对象。 | |||
| ### 6. 删除令牌 | |||
| ``` | |||
| DELETE /api/token/:id | |||
| ``` | |||
| 软删除,只能删除自己的 token。 | |||
| ### 7. 批量删除令牌 | |||
| ``` | |||
| POST /api/token/batch | |||
| ``` | |||
| #### 请求体 | |||
| ```json | |||
| {"ids":[1,2,3]} | |||
| ``` | |||
| #### 响应 | |||
| `data` 为实际删除数量。 | |||
| ### 8. 查询令牌用量 | |||
| ``` | |||
| GET /api/usage/token/ | |||
| ``` | |||
| #### 请求头 | |||
| ```bash | |||
| Authorization: Bearer sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | |||
| ``` | |||
| #### 响应 | |||
| ```json | |||
| { | |||
| "code": true, | |||
| "message": "ok", | |||
| "data": { | |||
| "object": "token_usage", | |||
| "name": "my-api-key", | |||
| "total_granted": 620000, | |||
| "total_used": 120000, | |||
| "total_available": 500000, | |||
| "unlimited_quota": false, | |||
| "model_limits": { | |||
| "gpt-4o": true | |||
| }, | |||
| "model_limits_enabled": true, | |||
| "expires_at": 0 | |||
| } | |||
| } | |||
| ``` | |||
| `expires_at` 单位是秒,`0` 表示永不过期。 | |||
| ## 错误消息速查 | |||
| | 场景 | message | | |||
| |------|---------| | |||
| | 未提供 `New-Api-User` | `无权进行此操作,未提供 New-Api-User` | | |||
| | `New-Api-User` 格式错误 | `无权进行此操作,New-Api-User 格式错误` | | |||
| | `New-Api-User` 不匹配 | `无权进行此操作,New-Api-User 与登录用户不匹配` | | |||
| | 名称过长 | `令牌名称过长` | | |||
| | 额度为负 | `额度值不能为负数` | | |||
| | 额度超限 | `额度值超出有效范围,最大值为 {{.Max}}` | | |||
| | 数量达上限 | `已达到最大令牌数量限制 (N)` | | |||
| | 生成失败 | `生成令牌失败` | | |||
| | 过期不能启用 | `令牌已过期,无法启用,请先修改令牌过期时间,或者设置为永不过期` | | |||
| | 耗尽不能启用 | `令牌可用额度已用尽,无法启用,请先修改令牌剩余额度,或者设置为无限额度` | | |||
| | 搜索通配符过多 | `搜索模式中最多允许包含 2 个 % 通配符` | | |||
| | 连续 `%` | `搜索模式中不允许包含连续的 % 通配符` | | |||
| | 模糊搜索太短 | `使用模糊搜索时,关键词长度至少为 2 个字符` | | |||
| | 超量用户模糊搜索 | `令牌数量超过上限,仅允许精确搜索,请勿使用 % 通配符` | | |||
| | 无效令牌 | `无效的令牌` | | |||
| | 令牌过期 | `该令牌已过期` | | |||
| | 令牌耗尽 | `该令牌额度已用尽 ...` | | |||
| ## 状态枚举 | |||
| | 值 | 常量名 | 含义 | | |||
| |----|--------|------| | |||
| | 1 | `TokenStatusEnabled` | 已启用 | | |||
| | 2 | `TokenStatusDisabled` | 已禁用 | | |||
| | 3 | `TokenStatusExpired` | 已过期 | | |||
| | 4 | `TokenStatusExhausted` | 已耗尽 | | |||
| @@ -0,0 +1,161 @@ | |||
| # 登录 Widget 使用指南 | |||
| ## 概述 | |||
| 登录 Widget 是一个轻量 Headless SDK(0.8KB),通过 `login()` 函数调用 New API 登录接口。不注入任何 DOM 或样式,你完全控制 UI。 | |||
| **仅支持密码登录。** | |||
| --- | |||
| ## 快速开始 | |||
| ```html | |||
| <script src="https://lancerouter.ai/static/login-widget.js"></script> | |||
| <script> | |||
| const result = await NewApiLoginWidget.login({ | |||
| apiUrl: 'https://lancerouter.ai', | |||
| username: 'user', | |||
| password: 'pass' | |||
| }); | |||
| if (result.success) { | |||
| console.log('登录成功', result.data); | |||
| } else { | |||
| console.log('登录失败', result.message); | |||
| } | |||
| </script> | |||
| ``` | |||
| --- | |||
| ## API 参考 | |||
| ### `NewApiLoginWidget.login(options)` | |||
| | 参数 | 类型 | 必填 | 说明 | | |||
| |------|------|------|------| | |||
| | `apiUrl` | `string` | 是 | New API 服务地址 | | |||
| | `username` | `string` | 是 | 用户名或邮箱 | | |||
| | `password` | `string` | 是 | 密码 | | |||
| **返回值:** `Promise<{ success: boolean, data?: object, message?: string }>` | |||
| 成功: | |||
| ```json | |||
| { | |||
| "success": true, | |||
| "data": { | |||
| "id": 1, | |||
| "username": "user", | |||
| "display_name": "用户", | |||
| "role": 1, | |||
| "status": 1, | |||
| "group": "default" | |||
| } | |||
| } | |||
| ``` | |||
| 失败: | |||
| ```json | |||
| { | |||
| "success": false, | |||
| "message": "用户名或密码错误,或用户已被封禁" | |||
| } | |||
| ``` | |||
| ### data 字段 | |||
| | 字段 | 类型 | 说明 | | |||
| |------|------|------| | |||
| | `id` | number | 用户 ID,调用其他 API 时需要 | | |||
| | `username` | string | 用户名 | | |||
| | `display_name` | string | 显示名称 | | |||
| | `role` | number | 角色(1=普通,10+=管理员,100+=超管) | | |||
| | `status` | number | 状态 | | |||
| | `group` | string | 用户组 | | |||
| --- | |||
| ## 登录后调用其他 API | |||
| 登录成功后浏览器会自动存储 session cookie,后续请求带 `credentials: 'include'` 和 `New-Api-User` 头即可。 | |||
| ### 获取用户信息 | |||
| ```js | |||
| const res = await fetch('https://lancerouter.ai/api/user/self', { | |||
| credentials: 'include', | |||
| headers: { 'New-Api-User': String(result.data.id) } | |||
| }); | |||
| const data = await res.json(); | |||
| ``` | |||
| ### 创建 API Token | |||
| ```js | |||
| const res = await fetch('https://lancerouter.ai/api/token/', { | |||
| method: 'POST', | |||
| credentials: 'include', | |||
| headers: { | |||
| 'Content-Type': 'application/json', | |||
| 'New-Api-User': String(result.data.id) | |||
| }, | |||
| body: JSON.stringify({ | |||
| name: 'my-token', | |||
| remain_quota: 1000000, | |||
| unlimited_quota: false | |||
| }) | |||
| }); | |||
| const data = await res.json(); | |||
| ``` | |||
| ### 完整示例 | |||
| ```html | |||
| <!DOCTYPE html> | |||
| <html> | |||
| <head> | |||
| <meta charset="UTF-8"> | |||
| <title>我的网站</title> | |||
| </head> | |||
| <body> | |||
| <form id="login-form"> | |||
| <input id="username" placeholder="用户名或邮箱" required> | |||
| <input id="password" type="password" placeholder="密码" required> | |||
| <button type="submit">登录</button> | |||
| </form> | |||
| <p id="msg"></p> | |||
| <script src="https://lancerouter.ai/static/login-widget.js"></script> | |||
| <script> | |||
| document.getElementById('login-form').addEventListener('submit', async (e) => { | |||
| e.preventDefault(); | |||
| const result = await NewApiLoginWidget.login({ | |||
| apiUrl: 'https://lancerouter.ai', | |||
| username: document.getElementById('username').value, | |||
| password: document.getElementById('password').value, | |||
| }); | |||
| if (result.success) { | |||
| localStorage.setItem('user', JSON.stringify(result.data)); | |||
| document.getElementById('msg').textContent = '欢迎, ' + result.data.username; | |||
| } else { | |||
| document.getElementById('msg').textContent = result.message; | |||
| } | |||
| }); | |||
| </script> | |||
| </body> | |||
| </html> | |||
| ``` | |||
| --- | |||
| ## 注意事项 | |||
| 1. **页面必须 HTTPS** — session cookie 带 `Secure` 标志,HTTP 页面下浏览器不会存储 | |||
| 2. **每个请求带 `New-Api-User` 头** — 值为登录返回的 `data.id`,否则接口返回未授权 | |||
| 3. **每个请求带 `credentials: 'include'`** — 否则浏览器不会携带 cookie | |||
| 4. **不支持 2FA** — 启用两步验证的账户会返回失败提示 | |||
| 5. **Cookie 有效期 30 天** | |||
| @@ -0,0 +1,648 @@ | |||
| # 重试机制技术文档 | |||
| > 本文档详细描述 new-api 的请求重试机制,包括重试流程、优先级降级、权重选择、状态码判断、跨组重试、渠道亲和性、计费退款、自动禁用等完整链路。 | |||
| --- | |||
| ## 目录 | |||
| 1. [整体流程概述](#1-整体流程概述) | |||
| 2. [重试次数与配置](#2-重试次数与配置) | |||
| 3. [优先级降级机制](#3-优先级降级机制) | |||
| 4. [权重选择机制](#4-权重选择机制) | |||
| 5. [重试判断规则](#5-重试判断规则) | |||
| 6. [可重试状态码范围](#6-可重试状态码范围) | |||
| 7. [Auto-Group 跨组重试](#7-auto-group-跨组重试) | |||
| 8. [渠道亲和性与重试](#8-渠道亲和性与重试) | |||
| 9. [计费与退款](#9-计费与退款) | |||
| 10. [自动禁用渠道](#10-自动禁用渠道) | |||
| 11. [Task Relay 重试](#11-task-relay-重试) | |||
| 12. [RetryParam 状态管理](#12-retryparam-状态管理) | |||
| --- | |||
| ## 1. 整体流程概述 | |||
| ### 请求完整链路 | |||
| ``` | |||
| HTTP Request | |||
| → router (SetApiRouter) | |||
| → middleware/auth.go (认证) | |||
| → middleware/distributor.go (初始渠道选择,retry=0) | |||
| → middleware/rate_limit.go (限流) | |||
| → controller/relay.go Relay() | |||
| → PreConsumeBilling() (预扣费) | |||
| → retry loop (retry=0 ~ RetryTimes): | |||
| → getChannel() (retry=0 用中间件选好的,retry≥1 重新选择) | |||
| → restore request body | |||
| → relayHandler() (发送到上游) | |||
| → 成功则 return | |||
| → 失败则 processChannelError() + shouldRetry() 判断 | |||
| → 全部失败 → BillingSession.Refund() (退款) | |||
| ``` | |||
| ### 关键文件与行号 | |||
| | 文件 | 函数/逻辑 | 行号 | | |||
| |------|----------|------| | |||
| | `controller/relay.go` | `Relay()` 重试循环 | 181-234 | | |||
| | `controller/relay.go` | `getChannel()` 渠道获取 | 283-313 | | |||
| | `controller/relay.go` | `shouldRetry()` 重试判断 | 315-339 | | |||
| | `controller/relay.go` | `processChannelError()` 错误处理 | 341-392 | | |||
| | `controller/relay.go` | `shouldRetryTaskRelay()` 任务重试判断 | 604-641 | | |||
| | `model/channel_cache.go` | `GetRandomSatisfiedChannel()` 优先级+权重选择 | 96-191 | | |||
| | `service/channel_select.go` | `CacheGetRandomSatisfiedChannel()` 跨组重试编排 | 83-162 | | |||
| | `service/channel_select.go` | `RetryParam` 结构体 | 14-46 | | |||
| | `middleware/distributor.go` | 初始渠道选择 | 60-137 | | |||
| | `setting/operation_setting/status_code_ranges.go` | 状态码范围配置 | 1-197 | | |||
| | `service/channel_affinity.go` | 渠道亲和性重试判断 | 493-532 | | |||
| | `service/billing_session.go` | 计费会话 | 1-115 | | |||
| | `service/channel.go` | `ShouldDisableChannel()` 自动禁用 | 47-102 | | |||
| ### 重试循环核心代码 | |||
| `controller/relay.go:181-234`: | |||
| ```go | |||
| retryParam := &service.RetryParam{ | |||
| Ctx: c, | |||
| TokenGroup: relayInfo.TokenGroup, | |||
| ModelName: relayInfo.OriginModelName, | |||
| Retry: common.GetPointer(0), | |||
| } | |||
| for ; retryParam.GetRetry() <= common.RetryTimes; retryParam.IncreaseRetry() { | |||
| channel, channelErr := getChannel(c, relayInfo, retryParam) | |||
| // ... 渠道选择、body 恢复、请求转发 ... | |||
| if newAPIError == nil { | |||
| return // 成功,直接返回 | |||
| } | |||
| processChannelError(c, ...) // 记录错误 + 可能自动禁用 | |||
| if !shouldRetry(c, newAPIError, common.RetryTimes-retryParam.GetRetry()) { | |||
| break // 不可重试,退出循环 | |||
| } | |||
| } | |||
| // 循环结束,defer 执行退款 | |||
| ``` | |||
| --- | |||
| ## 2. 重试次数与配置 | |||
| ### RetryTimes 配置 | |||
| | 属性 | 值 | | |||
| |------|---| | |||
| | 变量 | `common.RetryTimes` | | |||
| | 定义 | `common/constants.go:122` | | |||
| | 默认值 | **0**(不重试,只做 1 次初始尝试) | | |||
| | 存储位置 | 数据库 `options` 表,key=`RetryTimes` | | |||
| | 加载方式 | `model/option.go:496-497`,`strconv.Atoi(value)` | | |||
| | 配置入口 | 管理后台 → 运营设置 | | |||
| ### 循环次数计算 | |||
| ``` | |||
| 总尝试次数 = RetryTimes + 1 | |||
| ``` | |||
| 循环条件 `retryParam.GetRetry() <= common.RetryTimes`: | |||
| | RetryTimes 值 | 循环范围 | 总尝试次数 | | |||
| |:---:|:---:|:---:| | |||
| | 0 | retry=0 | 1 次(仅初始尝试) | | |||
| | 1 | retry=0, 1 | 2 次 | | |||
| | 3 | retry=0, 1, 2, 3 | 4 次 | | |||
| | 5 | retry=0, 1, 2, 3, 4, 5 | 6 次 | | |||
| ### 与优先级的关系 | |||
| `shouldRetry()` 的 `retryTimes` 参数 = `common.RetryTimes - retryParam.GetRetry()`,表示剩余重试次数。当 `retryTimes <= 0` 时不会重试。 | |||
| --- | |||
| ## 3. 优先级降级机制 | |||
| ### 核心算法 | |||
| `model/channel_cache.go:125-142`: | |||
| ```go | |||
| // 1. 收集所有唯一优先级值 | |||
| uniquePriorities := make(map[int]bool) | |||
| for _, channelId := range channels { | |||
| uniquePriorities[int(channel.GetPriority())] = true | |||
| } | |||
| // 2. 降序排序(最高优先级在前) | |||
| sort.Sort(sort.Reverse(sort.IntSlice(sortedUniquePriorities))) | |||
| // 3. retry index 映射到优先级层级 | |||
| if retry >= len(uniquePriorities) { | |||
| retry = len(uniquePriorities) - 1 // 限制为最低优先级 | |||
| } | |||
| targetPriority := int64(sortedUniquePriorities[retry]) | |||
| ``` | |||
| ### 映射规则 | |||
| ``` | |||
| retry=0 → 最高优先级(sortedUniquePriorities[0]) | |||
| retry=1 → 次高优先级(sortedUniquePriorities[1]) | |||
| ... | |||
| retry ≥ 层级数 → 降为最低优先级(不再降级) | |||
| ``` | |||
| ### 示例场景 | |||
| 假设某模型有 5 个渠道: | |||
| | 渠道 | 优先级 | 权重 | | |||
| |------|:---:|:---:| | |||
| | A | 100 | 50 | | |||
| | B | 100 | 30 | | |||
| | C | 50 | 80 | | |||
| | D | 10 | 20 | | |||
| | E | 10 | 10 | | |||
| 唯一优先级(降序):`[100, 50, 10]` | |||
| | 尝试 | retry 值 | 选择优先级 | 候选渠道 | | |||
| |:---:|:---:|:---:|:---:| | |||
| | 第 1 次 | 0 | 100 | A, B | | |||
| | 第 2 次 | 1 | 50 | C | | |||
| | 第 3 次 | 2 | 10 | D, E | | |||
| | 第 4 次+ | 超出范围,降为 2(最低优先级) | 10 | D, E | | |||
| --- | |||
| ## 4. 权重选择机制 | |||
| ### 加权随机算法 | |||
| `model/channel_cache.go:145-188`: | |||
| 在同一优先级的候选渠道中,使用加权随机选择: | |||
| ```go | |||
| totalWeight := sumWeight * smoothingFactor | |||
| randomWeight := rand.Intn(totalWeight) | |||
| for _, channel := range targetChannels { | |||
| randomWeight -= channel.GetWeight()*smoothingFactor + smoothingAdjustment | |||
| if randomWeight < 0 { | |||
| return channel, nil | |||
| } | |||
| } | |||
| ``` | |||
| ### 权重特殊处理 | |||
| `model/channel_cache.go:162-174`: | |||
| | 场景 | 处理 | | |||
| |------|------| | |||
| | 所有渠道 weight=0 | 每个 channel 的 effective weight = 100(均等选择) | | |||
| | 平均 weight < 10 | 乘以 smoothingFactor=100,提高区分度 | | |||
| | 正常 weight | 直接使用配置的 weight 值 | | |||
| ### 示例 | |||
| 渠道 A(weight=70)和 渠道 B(weight=30)在同一优先级: | |||
| - totalWeight = 100 | |||
| - randomWeight ∈ [0, 100) | |||
| - A 被选中的概率 = 70% | |||
| - B 被选中的概率 = 30% | |||
| E2E 测试验证:30 次请求中,21 次到 A(70.0%),9 次到 B(30.0%),与配置完全一致。 | |||
| --- | |||
| ## 5. 重试判断规则 | |||
| ### shouldRetry() 完整判断链 | |||
| `controller/relay.go:315-339`: | |||
| ``` | |||
| shouldRetry(c, openaiErr, retryTimes) | |||
| │ | |||
| ├─ openaiErr == nil? → 不重试 (false) | |||
| │ | |||
| ├─ 渠道亲和性跳过? → 不重试 (false) | |||
| │ ShouldSkipRetryAfterChannelAffinityFailure(c) | |||
| │ | |||
| ├─ 是 channel error? → 必须重试 (true) ← 优先级最高 | |||
| │ IsChannelError(openaiErr) 错误码以 "channel:" 开头 | |||
| │ | |||
| ├─ 是 skip-retry error? → 不重试 (false) | |||
| │ IsSkipRetryError(openaiErr) skipRetry=true 标记 | |||
| │ | |||
| ├─ retryTimes <= 0? → 不重试 (false) | |||
| │ 剩余重试次数用完 | |||
| │ | |||
| ├─ 2xx 状态码? → 不重试 (false) | |||
| │ code >= 200 && code < 300 | |||
| │ | |||
| ├─ 非 HTTP 状态码? → 重试 (true) | |||
| │ code < 100 || code > 599 连接/传输层错误 | |||
| │ | |||
| └─ 按状态码范围判断 → ShouldRetryByStatusCode(code) | |||
| ``` | |||
| ### Channel Error(必重试) | |||
| 以下错误码**始终触发重试**,无论其他条件: | |||
| | 错误码 | 说明 | | |||
| |--------|------| | |||
| | `channel:no_available_key` | 渠道无可用 API Key | | |||
| | `channel:param_override_invalid` | 参数覆写无效 | | |||
| | `channel:header_override_invalid` | Header 覆写无效 | | |||
| | `channel:model_mapped_error` | 模型映射错误 | | |||
| | `channel:aws_client_error` | AWS 客户端错误 | | |||
| | `channel:invalid_key` | 无效的 API Key | | |||
| | `channel:response_time_exceeded` | 响应超时 | | |||
| ### Skip-Retry Error(不重试) | |||
| 以下场景标记为 `skipRetry=true`,不触发重试: | |||
| - 请求体读取失败 | |||
| - 渠道未找到 | |||
| - 计费/预扣费失败 | |||
| - 敏感词检测 | |||
| - 各种内部错误 | |||
| --- | |||
| ## 6. 可重试状态码范围 | |||
| ### 默认配置 | |||
| `setting/operation_setting/status_code_ranges.go:19-32`: | |||
| ```go | |||
| AutomaticRetryStatusCodeRanges = []StatusCodeRange{ | |||
| {Start: 100, End: 199}, // 1xx 信息响应 | |||
| {Start: 300, End: 399}, // 3xx 重定向 | |||
| {Start: 401, End: 407}, // 认证错误 | |||
| {Start: 409, End: 499}, // 409-499 客户端错误 | |||
| {Start: 500, End: 503}, // 500-503 服务端错误 | |||
| {Start: 505, End: 523}, // 505-523 服务端错误 | |||
| {Start: 525, End: 599}, // 525-599 服务端错误 | |||
| } | |||
| alwaysSkipRetryStatusCodes = {504, 524} // 超时不重试 | |||
| ``` | |||
| ### 完整状态码速查表 | |||
| | 状态码范围 | 是否重试 | 说明 | | |||
| |-----------|:---:|------| | |||
| | 100-199 | 重试 | 信息响应 | | |||
| | 200-299 | **不重试** | 成功响应 | | |||
| | 300-399 | 重试 | 重定向 | | |||
| | 400 | **不重试** | 不在范围内 | | |||
| | 401-407 | 重试 | 认证错误 | | |||
| | 408 | **不重试** | Request Timeout(不在范围内) | | |||
| | 409-499 | 重试 | 各种客户端错误 | | |||
| | 500-503 | 重试 | 服务端错误 | | |||
| | **504** | **不重试** | Gateway Timeout(always-skip) | | |||
| | 505-523 | 重试 | 服务端错误 | | |||
| | **524** | **不重试** | Cloudflare Timeout(always-skip) | | |||
| | 525-599 | 重试 | 服务端错误 | | |||
| | <100 或 >599 | 重试 | 非 HTTP 状态(传输层错误) | | |||
| ### 自定义配置 | |||
| 管理员可通过运营设置修改 `AutomaticRetryStatusCodeRanges`,格式为逗号分隔的范围: | |||
| ``` | |||
| "500-503,525-599" # 范围表示 | |||
| "429" # 单个状态码 | |||
| "500-503,429,525-599" # 混合使用 | |||
| ``` | |||
| 解析函数 `ParseHTTPStatusCodeRanges()` 会自动排序、合并重叠范围。 | |||
| --- | |||
| ## 7. Auto-Group 跨组重试 | |||
| ### 触发条件 | |||
| 当 Token 的 `group` 字段为 `"auto"` 时启用。系统会在多个渠道组之间依次尝试,每个组内耗尽所有优先级后才切换到下一个组。 | |||
| ### 跨组重试开关 | |||
| 需要在 Token 上启用 `cross_group_retry`,存储在 context key `ContextKeyTokenCrossGroupRetry` 中。**未启用时只在第一个匹配的组内重试。** | |||
| ### 状态跟踪 | |||
| 通过 Gin Context 中的三个 key 跟踪状态: | |||
| | Context Key | 说明 | | |||
| |-------------|------| | |||
| | `ContextKeyAutoGroupIndex` | 当前正在尝试的组索引 | | |||
| | `ContextKeyAutoGroupRetryIndex` | 切换到当前组时的全局 retry 计数 | | |||
| | `ContextKeyAutoGroup` | 当前选中的组名 | | |||
| ### 示例流程 | |||
| 2 个组,每组 2 个优先级,`RetryTimes=3`: | |||
| ``` | |||
| Retry=0: GroupA, priority0 (startRetryIndex=0, priorityRetry=0) | |||
| → GroupA 最高优先级渠道 | |||
| Retry=1: GroupA, priority1 (startRetryIndex=0, priorityRetry=1) | |||
| → GroupA 次高优先级渠道 | |||
| Retry=2: GroupA 优先级用完 → GroupB, priority0 (startRetryIndex=2, priorityRetry=0) | |||
| → GroupB 最高优先级渠道 | |||
| Retry=3: GroupB, priority1 (startRetryIndex=2, priorityRetry=1) | |||
| → GroupB 次高优先级渠道 | |||
| ``` | |||
| ### 组切换逻辑 | |||
| `service/channel_select.go:106-154`: | |||
| ```go | |||
| for i := startGroupIndex; i < len(autoGroups); i++ { | |||
| priorityRetry := param.GetRetry() | |||
| if i > startGroupIndex { | |||
| priorityRetry = 0 // 新组从最高优先级开始 | |||
| } | |||
| channel, _ = model.GetRandomSatisfiedChannel(autoGroup, param.ModelName, priorityRetry) | |||
| if channel == nil { | |||
| // 当前组没有该模型/优先级的可用渠道,切换到下一组 | |||
| common.SetContextKey(param.Ctx, constant.ContextKeyAutoGroupIndex, i+1) | |||
| param.SetRetry(0) | |||
| continue | |||
| } | |||
| // 跨组重试:当前组重试次数耗尽,准备切换 | |||
| if crossGroupRetry && priorityRetry >= common.RetryTimes { | |||
| common.SetContextKey(param.Ctx, constant.ContextKeyAutoGroupIndex, i+1) | |||
| param.SetRetry(0) | |||
| param.ResetRetryNextTry() // 防止外层循环再 +1 | |||
| } | |||
| break | |||
| } | |||
| ``` | |||
| --- | |||
| ## 8. 渠道亲和性与重试 | |||
| ### 亲和性原理 | |||
| 渠道亲和性(Channel Affinity)是一种缓存机制:对于相同用户+模型的请求,优先使用上次成功的渠道,以保持会话连续性(如 Claude Code 的对话上下文)。 | |||
| ### 对重试的影响 | |||
| `service/channel_affinity.go:493-506`: | |||
| 当亲和性选择的渠道请求失败时,可能**跳过重试**: | |||
| ``` | |||
| ShouldSkipRetryAfterChannelAffinityFailure(c) | |||
| → 检查 ginKeyChannelAffinitySkipRetry 标记 | |||
| → true = 不重试(直接返回错误) | |||
| → false = 正常重试(默认行为) | |||
| ``` | |||
| ### 亲和性标记设置 | |||
| `service/channel_affinity.go:508-532`: | |||
| ```go | |||
| func MarkChannelAffinityUsed(c *gin.Context, selectedGroup string, channelID int) { | |||
| // ... | |||
| c.Set(ginKeyChannelAffinitySkipRetry, meta.SkipRetry) | |||
| // ... | |||
| } | |||
| ``` | |||
| 标记值来自亲和性规则的 `SkipRetryOnFailure` 字段。 | |||
| ### 默认规则 | |||
| | 规则名 | 匹配模型 | 端点 | 亲和性 Key | SkipRetryOnFailure | | |||
| |--------|---------|------|-----------|:---:| | |||
| | codex trace | `^gpt-.*$` | `/v1/responses` | `prompt_cache_key` | **false** | | |||
| | claude code trace | `^claude-.*$` | `/v1/messages` | `metadata.user_id` | **false** | | |||
| 默认两条规则都**不阻止重试**。只有管理员创建了 `SkipRetryOnFailure=true` 的规则时,亲和性失败才会跳过重试。 | |||
| ### 亲和性记录时机 | |||
| - **记录**:请求成功(status < 400)后,`middleware/distributor.go:135` 调用 `RecordChannelAffinity()` | |||
| - **查询**:初始渠道选择时,`distributor.go:86-107` 调用 `GetPreferredChannelByAffinity()` | |||
| --- | |||
| ## 9. 计费与退款 | |||
| ### 计费生命周期 | |||
| ``` | |||
| PreConsumeBilling() ← 预扣费(retry 循环之前) | |||
| │ | |||
| ├─ Retry 0: 尝试请求 ─── 成功 → Settle() 结算 → 结束 | |||
| │ └─ 失败 → 继续重试 | |||
| ├─ Retry 1: 尝试请求 ─── 成功 → Settle() 结算 → 结束 | |||
| │ └─ 失败 → 继续重试 | |||
| ├─ ... | |||
| │ | |||
| └─ 全部失败 → BillingSession.Refund() ← defer 函数,只执行一次 | |||
| ``` | |||
| ### 关键特性 | |||
| 1. **预扣费只一次**:在 retry 循环开始前执行 `PreConsumeBilling()`(`controller/relay.go:164`) | |||
| 2. **多次重试复用同一 BillingSession**:不重复扣费 | |||
| 3. **退款只一次**:通过 `defer` 函数(`controller/relay.go:170-179`),仅在 `newAPIError != nil` 时触发 | |||
| 4. **退款幂等**:`BillingSession.Refund()` 检查 `settled`、`refunded`、`fundingSettled` 标记 | |||
| 5. **异步执行**:退款操作通过 `gopool.Go()` 异步执行 | |||
| ### 退款流程代码 | |||
| ```go | |||
| defer func() { | |||
| if newAPIError != nil { | |||
| newAPIError = service.NormalizeViolationFeeError(newAPIError) | |||
| if relayInfo.Billing != nil { | |||
| relayInfo.Billing.Refund(c) // 退还预扣额度 | |||
| } | |||
| service.ChargeViolationFeeIfNeeded(c, relayInfo, newAPIError) | |||
| } | |||
| }() | |||
| ``` | |||
| ### 信任用户优化 | |||
| 高余额用户(Trust Quota Bypass)跳过预扣费,`preConsumedQuota` 为 0,直接在 Settle 时扣费。 | |||
| --- | |||
| ## 10. 自动禁用渠道 | |||
| ### 触发时机 | |||
| 每次 relay 失败后调用 `processChannelError()`(`controller/relay.go:229`),其中包含自动禁用逻辑。 | |||
| ### 判断条件 | |||
| `service/channel.go:47-102` `ShouldDisableChannel()`: | |||
| | 条件 | 说明 | | |||
| |------|------| | |||
| | `AutomaticDisableChannelEnabled = false` | 全局开关关闭则不触发 | | |||
| | `IsChannelError(err)` | `channel:*` 错误码,始终触发 | | |||
| | `ShouldDisableByStatusCode(code)` | 默认 401 触发 | | |||
| | 403 + Gemini 渠道 | Gemini 特有 | | |||
| | 错误码匹配 | `invalid_api_key`、`account_deactivated`、`billing_not_active`、`Arrearage` 等 | | |||
| | 错误类型匹配 | `insufficient_quota`、`authentication_error`、`permission_error`、`forbidden` | | |||
| | 关键词匹配 | `AutomaticDisableKeywords` 中的关键词 | | |||
| ### 执行方式 | |||
| ```go | |||
| if service.ShouldDisableChannel(channelType, err) && channelError.AutoBan { | |||
| gopool.Go(func() { | |||
| service.DisableChannel(channelError, err.ErrorWithStatusCode()) | |||
| }) | |||
| } | |||
| ``` | |||
| - **异步执行**:通过 `gopool.Go()` 不阻塞请求处理 | |||
| - **AutoBan 控制**:每个渠道可配置是否允许自动禁用(`auto_ban` 字段) | |||
| - **自动恢复**:当 `AutomaticEnableChannelEnabled=true` 时,渠道后续请求成功会自动重新启用 | |||
| ### 自动禁用状态码范围 | |||
| ```go | |||
| AutomaticDisableStatusCodeRanges = []StatusCodeRange{{Start: 401, End: 401}} | |||
| ``` | |||
| 默认只有 401(未授权)触发自动禁用。可自定义。 | |||
| --- | |||
| ## 11. Task Relay 重试 | |||
| ### 适用场景 | |||
| 异步任务提交(Midjourney、Suno 等非实时请求),使用 `RelayTask()` 函数(`controller/relay.go:476`)。 | |||
| ### 独立的重试判断 | |||
| `shouldRetryTaskRelay()`(`controller/relay.go:604-641`)与主 `shouldRetry()` 的区别: | |||
| | 条件 | shouldRetry() | shouldRetryTaskRelay() | | |||
| |------|:---:|:---:| | |||
| | channel error | 必重试 | 不检查 | | |||
| | skip-retry 标记 | 不重试 | 不检查 | | |||
| | 429 | 按状态码范围判断 | **必重试** | | |||
| | 307 | 按状态码范围判断 | **必重试** | | |||
| | 5xx | 按状态码范围判断 | 重试(排除 504/524) | | |||
| | 400 | 不在默认范围,不重试 | **明确不重试** | | |||
| | 408 | 不在默认范围,不重试 | **明确不重试**(Azure 超时) | | |||
| | LocalError | 不适用 | **不重试** | | |||
| | 渠道亲和性 | 检查 | 检查 | | |||
| | 2xx | 不重试 | 不重试 | | |||
| | 其他 | 委托 ShouldRetryByStatusCode | **重试**(兜底) | | |||
| ### Locked Channel | |||
| Task Relay 支持 `LockedChannel`:如果 relayInfo 中有锁定的渠道,所有重试都使用该渠道(不切换)。 | |||
| --- | |||
| ## 12. RetryParam 状态管理 | |||
| ### 结构体定义 | |||
| `service/channel_select.go:14-20`: | |||
| ```go | |||
| type RetryParam struct { | |||
| Ctx *gin.Context | |||
| TokenGroup string | |||
| ModelName string | |||
| Retry *int // 当前重试计数 | |||
| resetNextTry bool // 标记下一次 IncreaseRetry 是否跳过 | |||
| } | |||
| ``` | |||
| ### 方法 | |||
| | 方法 | 说明 | | |||
| |------|------| | |||
| | `GetRetry() int` | 返回当前 retry 值,nil 时返回 0 | | |||
| | `SetRetry(retry int)` | 直接设置 retry 值 | | |||
| | `IncreaseRetry()` | retry + 1;若 `resetNextTry=true` 则跳过并重置标记 | | |||
| | `ResetRetryNextTry()` | 设置 `resetNextTry=true` | | |||
| ### resetNextTry 机制 | |||
| 用于 Auto-Group 跨组重试时的状态管理,防止 retry 计数器被双重递增: | |||
| ``` | |||
| 场景:GroupA 重试用完,准备切换到 GroupB | |||
| 1. CacheGetRandomSatisfiedChannel 检测到 priorityRetry >= RetryTimes | |||
| 2. 调用 param.SetRetry(0) ← 重置计数器 | |||
| 3. 调用 param.ResetRetryNextTry() ← 标记下次跳过 | |||
| 4. 外层 for 循环调用 IncreaseRetry() | |||
| 5. 检测到 resetNextTry=true → 跳过递增,重置标记 | |||
| 6. 结果:retry 保持为 0,GroupB 从最高优先级开始 | |||
| ``` | |||
| 如果不跳过,retry 会从 0 变成 1,导致 GroupB 直接从次高优先级开始,跳过了最高优先级的渠道。 | |||
| ### Context Key | |||
| `constant/context_key.go`: | |||
| ```go | |||
| ContextKeyAutoGroup // 当前 auto-group 名称 | |||
| ContextKeyAutoGroupIndex // 当前组索引 | |||
| ContextKeyAutoGroupRetryIndex // 当前组开始时的全局 retry 数 | |||
| ContextKeyTokenCrossGroupRetry // 是否启用跨组重试 | |||
| ``` | |||
| --- | |||
| ## 附录:重试链路追踪 | |||
| ### 日志 | |||
| 重试过程中使用了多个渠道时,循环结束后会打印重试链路: | |||
| ```go | |||
| useChannel := c.GetStringSlice("use_channel") | |||
| if len(useChannel) > 1 { | |||
| retryLogStr := fmt.Sprintf("重试:%s", strings.Trim(...)) | |||
| logger.LogInfo(c, retryLogStr) | |||
| } | |||
| ``` | |||
| 输出示例:`重试:5->12->8`(渠道 ID 5 失败后重试到 12,再到 8)。 | |||
| ### 错误日志 | |||
| 每次失败的渠道信息都会记录到数据库 `logs` 表(需开启 `ErrorLogEnabled`),包含: | |||
| - 渠道 ID、名称、类型 | |||
| - 状态码、错误类型、错误码 | |||
| - 上游请求 ID | |||
| - `use_channel` 数组(重试链路) | |||
| - 渠道亲和性信息 | |||
| @@ -0,0 +1,12 @@ | |||
| | Commit | 问题 | 位置 | 影响 | | |||
| |---|---|---|---| | |||
| | `1cc75aa9` / `8bb2883d` | 法律页路由重复,`Terms` / `UsagePolicy` 不可达 | `web/src/App.jsx:359-372` | 页面实际不会渲染新组件 | | |||
| | `67e78f2c` | Metrics label 全为空字符串 | `common/metrics/middleware.go:14-22` | 指标维度丢失 | | |||
| | `123f0d95` | `/metrics` 直接暴露,无鉴权 | `router/relay-router.go:23-25` | 运维数据可被未授权访问 | | |||
| | `1912dd3b` | 验证码只保护发邮箱验证码,不保护注册本身 | `controller/user.go:133-161` | 开关语义不一致,注册可绕过 | | |||
| | `f7ff684d` | `formatDiscount()` 对 `0.95` 返回空串 | `controller/pricing.go:176-186` | 折扣显示异常 | | |||
| | `967309fb` | 邮箱后缀规则大小写不一致导致重复/覆盖 | `model/email_quota_rule.go:10-18, 25-55, 126-132` | DB 可存重复规则,缓存会互相覆盖 | | |||
| | `714d8239` | `capture_relay` 明文落盘请求头/body | `middleware/relay_capture.go:101-127` | API Key / prompt / 上传内容泄露风险 | | |||
| | `714d8239` | 非 Redis 模式下用户模型 RPM 可能沿用旧值 | `middleware/user-model-rate-limit.go:25-55`, `controller/user_rate_limit.go:58-66` | 配置更新后短时间不生效 | | |||
| | `79f8a18c` | login widget 仍依赖 `credentials: 'include'` | `web/widget/index.js:26-30` | 仍受跨站 Cookie 限制 | | |||
| | `54573991` / `49c67556` | 头部导航 `docs/about` 是死配置 | `web/src/pages/Setting/Operation/SettingsHeaderNavModules.jsx:220-237`, `web/src/hooks/common/useNavigation.js:33-70` | 后台能配,前台不生效 | | |||
| @@ -0,0 +1,596 @@ | |||
| # 登录 Widget 实施计划 | |||
| > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. | |||
| **Goal:** 将现有前端登录表单封装为独立的 JS Widget,嵌入方通过 `<script>` 标签引入后渲染登录表单,调用现有 `/api/user/login` 接口完成认证。 | |||
| **Architecture:** 在 `web/widget/` 下创建独立的 Vite 构建,打包成 UMD 格式,全局暴露 `NewApiLoginWidget`。Widget 使用 Shadow DOM 隔离样式,内部用 React 渲染登录表单。React 和 ReactDOM 打包进 bundle,嵌入方无需任何依赖。零后端改动。 | |||
| **Tech Stack:** React 18, Vite (UMD output), Shadow DOM, fetch API | |||
| --- | |||
| ## File Structure | |||
| ``` | |||
| web/widget/ | |||
| package.json # 独立的 package.json(仅 React 依赖) | |||
| vite.config.js # UMD 打包配置,输出到 ../dist/static/ | |||
| index.jsx # 入口:init() / destroy() API,创建 Shadow DOM | |||
| LoginForm.jsx # 登录表单组件 | |||
| styles.css # Widget 样式 | |||
| ``` | |||
| --- | |||
| ### Task 1: 创建 Widget 项目骨架 | |||
| **Files:** | |||
| - Create: `web/widget/package.json` | |||
| - Create: `web/widget/vite.config.js` | |||
| - [ ] **Step 1: 创建 package.json** | |||
| `web/widget/package.json`: | |||
| ```json | |||
| { | |||
| "name": "new-api-login-widget", | |||
| "version": "1.0.0", | |||
| "private": true, | |||
| "type": "module", | |||
| "scripts": { | |||
| "dev": "vite", | |||
| "build": "vite build" | |||
| }, | |||
| "dependencies": { | |||
| "react": "^18.2.0", | |||
| "react-dom": "^18.2.0" | |||
| }, | |||
| "devDependencies": { | |||
| "@vitejs/plugin-react": "^4.2.1", | |||
| "vite": "^5.2.0" | |||
| } | |||
| } | |||
| ``` | |||
| - [ ] **Step 2: 创建 vite.config.js** | |||
| `web/widget/vite.config.js`: | |||
| ```js | |||
| import react from '@vitejs/plugin-react'; | |||
| import { defineConfig } from 'vite'; | |||
| import { resolve } from 'path'; | |||
| export default defineConfig({ | |||
| plugins: [react()], | |||
| build: { | |||
| lib: { | |||
| entry: resolve(__dirname, 'index.jsx'), | |||
| name: 'NewApiLoginWidget', | |||
| formats: ['umd'], | |||
| fileName: () => 'login-widget.js', | |||
| }, | |||
| outDir: resolve(__dirname, '../dist/static'), | |||
| emptyOutDir: false, | |||
| rollupOptions: { | |||
| external: [], | |||
| output: { | |||
| globals: {}, | |||
| }, | |||
| }, | |||
| }, | |||
| }); | |||
| ``` | |||
| 关键点: | |||
| - `name: 'NewApiLoginWidget'` — UMD 全局变量名 | |||
| - `outDir` 指向 `web/dist/static/`,产物为 `web/dist/static/login-widget.js` | |||
| - `emptyOutDir: false` — 不清空 dist 目录(那是主前端构建产物) | |||
| - React 和 ReactDOM 不做 external,打包进 bundle | |||
| - [ ] **Step 3: 安装依赖** | |||
| ```bash | |||
| cd web/widget && bun install | |||
| ``` | |||
| - [ ] **Step 4: 提交** | |||
| ```bash | |||
| git add web/widget/package.json web/widget/vite.config.js web/widget/bun.lock | |||
| git commit -m "feat(widget): 添加 Widget 项目骨架和 Vite 构建配置" | |||
| ``` | |||
| --- | |||
| ### Task 2: 创建 Widget 样式 | |||
| **Files:** | |||
| - Create: `web/widget/styles.css` | |||
| - [ ] **Step 1: 创建样式文件** | |||
| `web/widget/styles.css`: | |||
| ```css | |||
| @import url('https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600&display=swap'); | |||
| :host { | |||
| --naw-primary: #4f46e5; | |||
| --naw-primary-hover: #4338ca; | |||
| --naw-bg: #ffffff; | |||
| --naw-text: #1f2937; | |||
| --naw-text-secondary: #6b7280; | |||
| --naw-border: #d1d5db; | |||
| --naw-input-bg: #f9fafb; | |||
| --naw-error: #ef4444; | |||
| --naw-radius: 8px; | |||
| --naw-font: 'Inter', -apple-system, BlinkMacSystemFont, sans-serif; | |||
| display: block; | |||
| font-family: var(--naw-font); | |||
| } | |||
| :host([theme="dark"]) { | |||
| --naw-primary: #818cf8; | |||
| --naw-primary-hover: #6366f1; | |||
| --naw-bg: #1f2937; | |||
| --naw-text: #f9fafb; | |||
| --naw-text-secondary: #9ca3af; | |||
| --naw-border: #374151; | |||
| --naw-input-bg: #374151; | |||
| --naw-error: #f87171; | |||
| } | |||
| .naw-container { | |||
| max-width: 400px; | |||
| margin: 0 auto; | |||
| padding: 32px 24px; | |||
| background: var(--naw-bg); | |||
| border-radius: 16px; | |||
| } | |||
| .naw-title { | |||
| text-align: center; | |||
| font-size: 24px; | |||
| font-weight: 600; | |||
| color: var(--naw-text); | |||
| margin: 0 0 32px 0; | |||
| } | |||
| .naw-form { | |||
| display: flex; | |||
| flex-direction: column; | |||
| gap: 16px; | |||
| } | |||
| .naw-field { | |||
| display: flex; | |||
| flex-direction: column; | |||
| gap: 6px; | |||
| } | |||
| .naw-label { | |||
| font-size: 14px; | |||
| font-weight: 500; | |||
| color: var(--naw-text); | |||
| } | |||
| .naw-input { | |||
| width: 100%; | |||
| box-sizing: border-box; | |||
| padding: 10px 14px; | |||
| font-size: 15px; | |||
| border: 1px solid var(--naw-border); | |||
| border-radius: var(--naw-radius); | |||
| background: var(--naw-input-bg); | |||
| color: var(--naw-text); | |||
| outline: none; | |||
| transition: border-color 0.2s; | |||
| font-family: var(--naw-font); | |||
| } | |||
| .naw-input:focus { | |||
| border-color: var(--naw-primary); | |||
| } | |||
| .naw-input::placeholder { | |||
| color: var(--naw-text-secondary); | |||
| } | |||
| .naw-button { | |||
| width: 100%; | |||
| padding: 12px; | |||
| font-size: 15px; | |||
| font-weight: 600; | |||
| color: #ffffff; | |||
| background: var(--naw-primary); | |||
| border: none; | |||
| border-radius: var(--naw-radius); | |||
| cursor: pointer; | |||
| transition: background 0.2s; | |||
| font-family: var(--naw-font); | |||
| } | |||
| .naw-button:hover { | |||
| background: var(--naw-primary-hover); | |||
| } | |||
| .naw-button:disabled { | |||
| opacity: 0.6; | |||
| cursor: not-allowed; | |||
| } | |||
| .naw-error-msg { | |||
| color: var(--naw-error); | |||
| font-size: 13px; | |||
| text-align: center; | |||
| margin: 0; | |||
| min-height: 20px; | |||
| } | |||
| ``` | |||
| 关键点: | |||
| - 使用 `:host` 和 CSS 变量实现主题切换(light/dark) | |||
| - 所有类名以 `naw-` 前缀避免冲突(Shadow DOM 内其实不需要,但保持好习惯) | |||
| - 不依赖任何外部 UI 库(Semi UI 等),纯原生 CSS | |||
| - [ ] **Step 2: 提交** | |||
| ```bash | |||
| git add web/widget/styles.css | |||
| git commit -m "feat(widget): 添加 Widget 样式(light/dark 主题)" | |||
| ``` | |||
| --- | |||
| ### Task 3: 创建 LoginForm 组件 | |||
| **Files:** | |||
| - Create: `web/widget/LoginForm.jsx` | |||
| - [ ] **Step 1: 创建登录表单组件** | |||
| `web/widget/LoginForm.jsx`: | |||
| ```jsx | |||
| import React, { useState } from 'react'; | |||
| export default function LoginForm({ apiUrl, onSuccess, onError }) { | |||
| const [username, setUsername] = useState(''); | |||
| const [password, setPassword] = useState(''); | |||
| const [loading, setLoading] = useState(false); | |||
| const [error, setError] = useState(''); | |||
| const handleSubmit = async (e) => { | |||
| e.preventDefault(); | |||
| setError(''); | |||
| if (!username.trim() || !password.trim()) { | |||
| const msg = '请输入用户名和密码'; | |||
| setError(msg); | |||
| onError?.({ message: msg }); | |||
| return; | |||
| } | |||
| setLoading(true); | |||
| try { | |||
| const res = await fetch(`${apiUrl}/api/user/login`, { | |||
| method: 'POST', | |||
| headers: { 'Content-Type': 'application/json' }, | |||
| body: JSON.stringify({ username: username.trim(), password }), | |||
| credentials: 'include', | |||
| }); | |||
| const data = await res.json(); | |||
| if (data.success) { | |||
| if (data.data?.require_2fa) { | |||
| const msg = '此账户启用了两步验证,暂不支持通过 Widget 登录'; | |||
| setError(msg); | |||
| onError?.({ message: msg }); | |||
| return; | |||
| } | |||
| onSuccess?.(data.data); | |||
| } else { | |||
| const msg = data.message || '登录失败'; | |||
| setError(msg); | |||
| onError?.({ message: msg }); | |||
| } | |||
| } catch (err) { | |||
| const msg = '网络错误,请检查网络连接'; | |||
| setError(msg); | |||
| onError?.({ message: msg }); | |||
| } finally { | |||
| setLoading(false); | |||
| } | |||
| }; | |||
| return ( | |||
| <form className="naw-form" onSubmit={handleSubmit}> | |||
| <div className="naw-field"> | |||
| <label className="naw-label">用户名或邮箱</label> | |||
| <input | |||
| className="naw-input" | |||
| type="text" | |||
| placeholder="请输入用户名或邮箱地址" | |||
| value={username} | |||
| onChange={(e) => setUsername(e.target.value)} | |||
| autoComplete="username" | |||
| disabled={loading} | |||
| /> | |||
| </div> | |||
| <div className="naw-field"> | |||
| <label className="naw-label">密码</label> | |||
| <input | |||
| className="naw-input" | |||
| type="password" | |||
| placeholder="请输入密码" | |||
| value={password} | |||
| onChange={(e) => setPassword(e.target.value)} | |||
| autoComplete="current-password" | |||
| disabled={loading} | |||
| /> | |||
| </div> | |||
| <p className="naw-error-msg">{error}</p> | |||
| <button className="naw-button" type="submit" disabled={loading}> | |||
| {loading ? '登录中...' : '登录'} | |||
| </button> | |||
| </form> | |||
| ); | |||
| } | |||
| ``` | |||
| 关键点: | |||
| - 使用原生 `fetch` 而非 `axios`,减少 bundle 体积 | |||
| - `credentials: 'include'` 让浏览器携带/接收 Cookie(跨域需 CORS 支持,已配置) | |||
| - 2FA 场景给出提示而非静默失败 | |||
| - 通过 props 接收 `apiUrl`、`onSuccess`、`onError` | |||
| - [ ] **Step 2: 提交** | |||
| ```bash | |||
| git add web/widget/LoginForm.jsx | |||
| git commit -m "feat(widget): 添加 LoginForm 组件" | |||
| ``` | |||
| --- | |||
| ### Task 4: 创建 Widget 入口(Shadow DOM + init/destroy API) | |||
| **Files:** | |||
| - Create: `web/widget/index.jsx` | |||
| - [ ] **Step 1: 创建入口文件** | |||
| `web/widget/index.jsx`: | |||
| ```jsx | |||
| import React from 'react'; | |||
| import { createRoot } from 'react-dom/client'; | |||
| import LoginForm from './LoginForm'; | |||
| import cssText from './styles.css?inline'; | |||
| function mount(container, props) { | |||
| const host = document.createElement('div'); | |||
| host.setAttribute('data-new-api-widget', 'login'); | |||
| container.appendChild(host); | |||
| const shadow = host.attachShadow({ mode: 'open' }); | |||
| // 注入样式到 Shadow DOM | |||
| const style = document.createElement('style'); | |||
| style.textContent = cssText; | |||
| shadow.appendChild(style); | |||
| // 应用主题 | |||
| if (props.theme === 'dark') { | |||
| host.setAttribute('theme', 'dark'); | |||
| } | |||
| // 创建 React 挂载点 | |||
| const mountPoint = document.createElement('div'); | |||
| shadow.appendChild(mountPoint); | |||
| const root = createRoot(mountPoint); | |||
| root.render( | |||
| <div className="naw-container"> | |||
| <h1 className="naw-title">登 录</h1> | |||
| <LoginForm | |||
| apiUrl={props.apiUrl} | |||
| onSuccess={props.onSuccess} | |||
| onError={props.onError} | |||
| /> | |||
| </div> | |||
| ); | |||
| return { host, root }; | |||
| } | |||
| export function init(options) { | |||
| const container = | |||
| typeof options.container === 'string' | |||
| ? document.querySelector(options.container) | |||
| : options.container; | |||
| if (!container) { | |||
| throw new Error('NewApiLoginWidget: container not found'); | |||
| } | |||
| if (!options.apiUrl) { | |||
| throw new Error('NewApiLoginWidget: apiUrl is required'); | |||
| } | |||
| const { host, root } = mount(container, { | |||
| apiUrl: options.apiUrl.replace(/\/+$/, ''), | |||
| onSuccess: options.onSuccess, | |||
| onError: options.onError, | |||
| theme: options.theme || 'light', | |||
| }); | |||
| return { | |||
| destroy() { | |||
| root.unmount(); | |||
| host.remove(); | |||
| }, | |||
| }; | |||
| } | |||
| // UMD 全局导出 | |||
| if (typeof window !== 'undefined') { | |||
| window.NewApiLoginWidget = { init }; | |||
| } | |||
| ``` | |||
| 关键点: | |||
| - Shadow DOM 隔离样式,`mode: 'open'` 允许嵌入方检查 | |||
| - `?inline` 后缀让 Vite 将 CSS 作为字符串导入,注入到 Shadow DOM | |||
| - `container` 支持 CSS 选择器字符串或 DOM 元素 | |||
| - `apiUrl` 末尾斜杠清理 | |||
| - `destroy()` 卸载 React root 并移除 DOM 元素 | |||
| - UMD 全局暴露 `window.NewApiLoginWidget` | |||
| - [ ] **Step 2: 提交** | |||
| ```bash | |||
| git add web/widget/index.jsx | |||
| git commit -m "feat(widget): 添加 Widget 入口(Shadow DOM + init/destroy API)" | |||
| ``` | |||
| --- | |||
| ### Task 5: 构建验证 + 集成到构建流程 | |||
| **Files:** | |||
| - Modify: `Makefile` | |||
| - [ ] **Step 1: 构建 Widget** | |||
| ```bash | |||
| cd web/widget && bun run build | |||
| ``` | |||
| 预期产物:`web/dist/static/login-widget.js`(约 150-200KB,含 React) | |||
| - [ ] **Step 2: 验证产物** | |||
| ```bash | |||
| ls -la web/dist/static/login-widget.js | |||
| head -5 web/dist/static/login-widget.js | |||
| ``` | |||
| 预期:文件存在,前几行包含 UMD 包装器和 `NewApiLoginWidget`。 | |||
| - [ ] **Step 3: 更新 Makefile,将 Widget 构建加入主构建流程** | |||
| `Makefile` 修改: | |||
| ```makefile | |||
| FRONTEND_DIR = ./web | |||
| BACKEND_DIR = . | |||
| .PHONY: all build-frontend build-widget start-backend docker-build docker-push | |||
| all: build-frontend build-widget start-backend | |||
| build-frontend: | |||
| @echo "Building frontend..." | |||
| @cd $(FRONTEND_DIR) && bun install && DISABLE_ESLINT_PLUGIN='true' VITE_REACT_APP_VERSION=$(cat VERSION) bun run build | |||
| build-widget: | |||
| @echo "Building login widget..." | |||
| @cd $(FRONTEND_DIR)/widget && bun install && bun run build | |||
| start-backend: | |||
| @echo "Starting backend dev server..." | |||
| @cd $(BACKEND_DIR) && go run main.go & | |||
| ``` | |||
| - [ ] **Step 4: 提交** | |||
| ```bash | |||
| git add Makefile | |||
| git commit -m "feat(widget): 集成 Widget 构建到 Makefile" | |||
| ``` | |||
| --- | |||
| ### Task 6: 本地端到端验证 | |||
| **Files:** | |||
| - 无代码改动,纯验证 | |||
| - [ ] **Step 1: 启动本地 dev 服务器** | |||
| ```bash | |||
| cd D:/code/new-api && go run main.go | |||
| ``` | |||
| - [ ] **Step 2: 创建测试 HTML 页面** | |||
| 临时创建 `test-widget.html`: | |||
| ```html | |||
| <!DOCTYPE html> | |||
| <html> | |||
| <head><title>Widget 测试</title></head> | |||
| <body> | |||
| <h1>嵌入方页面测试</h1> | |||
| <div id="login-container"></div> | |||
| <script src="http://localhost:3000/static/login-widget.js"></script> | |||
| <script> | |||
| const widget = NewApiLoginWidget.init({ | |||
| container: '#login-container', | |||
| apiUrl: 'http://localhost:3000', | |||
| onSuccess: (data) => { | |||
| document.getElementById('result').textContent = JSON.stringify(data, null, 2); | |||
| widget.destroy(); | |||
| }, | |||
| onError: (err) => { | |||
| alert('错误: ' + err.message); | |||
| } | |||
| }); | |||
| </script> | |||
| <pre id="result"></pre> | |||
| </body> | |||
| </html> | |||
| ``` | |||
| - [ ] **Step 3: 浏览器打开测试页面,验证** | |||
| - 页面加载后应看到带用户名/密码输入框的登录表单 | |||
| - 输入错误密码应显示错误信息 | |||
| - 输入正确密码应触发 `onSuccess` 回调,显示用户数据 | |||
| - Shadow DOM 内样式应正常,不影响页面其他元素 | |||
| - [ ] **Step 4: 清理测试文件** | |||
| ```bash | |||
| rm test-widget.html | |||
| ``` | |||
| - [ ] **Step 5: 最终提交** | |||
| ```bash | |||
| git add web/widget/ web/dist/static/login-widget.js | |||
| git commit -m "feat(widget): 完成登录 Widget v1(密码登录,Shadow DOM 隔离)" | |||
| ``` | |||
| --- | |||
| ## Self-Review 检查 | |||
| ### 1. Spec 覆盖 | |||
| | Spec 要求 | 对应 Task | | |||
| |-----------|----------| | |||
| | UMD 打包,全局暴露 `NewApiLoginWidget` | Task 1 (vite.config.js), Task 4 (index.jsx) | | |||
| | `<script>` 标签引入 | Task 1 (UMD output), Task 5 (托管在 dist/static/) | | |||
| | `init({ container, apiUrl, onSuccess, onError, theme })` API | Task 4 | | |||
| | `destroy()` 方法 | Task 4 | | |||
| | Shadow DOM 样式隔离 | Task 4 | | |||
| | light/dark 主题 | Task 2 (CSS :host 变量), Task 4 (theme prop) | | |||
| | 调用现有 `POST /api/user/login` | Task 3 | | |||
| | 零后端改动 | 全局(无后端文件改动) | | |||
| | React 打包进 bundle | Task 1 (不 external React) | | |||
| | 构建流程集成 | Task 5 (Makefile) | | |||
| ### 2. Placeholder 扫描 | |||
| - 无 TBD、TODO、占位符 | |||
| ### 3. 类型一致性 | |||
| - `onSuccess` 回调参数统一为 `data.data`(API 响应中的 data 字段) | |||
| - `onError` 回调参数统一为 `{ message: string }` | |||
| - 所有文件中的 props 名称一致:`apiUrl`, `onSuccess`, `onError` | |||
| @@ -0,0 +1,932 @@ | |||
| # 扩展可视化倍率设置实施计划 | |||
| > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. | |||
| **目标:** 在 ModelSettingsVisualEditor 组件中添加 5 个高级倍率字段支持(缓存、图片、音频倍率) | |||
| **架构:** 扩展现有 React 组件的状态管理和表单逻辑,增加高级倍率字段的数据初始化、编辑、保存和验证功能 | |||
| **技术栈:** React, Semi UI, i18n, JavaScript | |||
| --- | |||
| ## 文件结构 | |||
| ### 修改文件 | |||
| - `web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx` - 主要功能实现(500+ 行) | |||
| - `web/src/i18n/locales/zh-CN.json` - 中文翻译 | |||
| - `web/src/i18n/locales/en.json` - 英文翻译 | |||
| ### 参考文件(只读,用于理解实现模式) | |||
| - `feat/channel-pricing-extended:web/src/pages/Setting/Ratio/ChannelPricingView.jsx` - 渠道价格设置的参考实现 | |||
| ### 数据流向 | |||
| ``` | |||
| props.options (API 数据) | |||
| ↓ | |||
| useEffect 解析 JSON → models 状态(包含 5 个新字段) | |||
| ↓ | |||
| 用户编辑表单 → currentModel 状态 | |||
| ↓ | |||
| addOrUpdateModel → 更新 models 状态 | |||
| ↓ | |||
| SubmitData → 序列化 JSON → API.put('/api/option/') | |||
| ``` | |||
| --- | |||
| ## Task 1: 添加国际化文本 | |||
| **文件:** | |||
| - Modify: `web/src/i18n/locales/zh-CN.json` | |||
| - Modify: `web/src/i18n/locales/en.json` | |||
| ### 步骤 1.1: 添加中文翻译 | |||
| 在 `web/src/i18n/locales/zh-CN.json` 中添加以下键值对。找到文件末尾,在最后一个键值对后添加逗号,然后添加: | |||
| ```json | |||
| "高级比例(留空使用默认值)": "高级比例(留空使用默认值)", | |||
| "缓存读取倍率": "缓存读取倍率", | |||
| "缓存创建倍率(5分钟)": "缓存创建倍率(5分钟)", | |||
| "图片倍率": "图片倍率", | |||
| "音频输入倍率": "音频输入倍率", | |||
| "音频输出倍率": "音频输出倍率", | |||
| "默认值:1.0": "默认值:1.0", | |||
| "默认值:1.25": "默认值:1.25" | |||
| ``` | |||
| ### 步骤 1.2: 添加英文翻译 | |||
| 在 `web/src/i18n/locales/en.json` 中添加对应的英文翻译: | |||
| ```json | |||
| "高级比例(留空使用默认值)": "Advanced Ratios (Leave empty for defaults)", | |||
| "缓存读取倍率": "Cache Read Ratio", | |||
| "缓存创建倍率(5分钟)": "Cache Creation Ratio (5min)", | |||
| "图片倍率": "Image Ratio", | |||
| "音频输入倍率": "Audio Input Ratio", | |||
| "音频输出倍率": "Audio Output Ratio", | |||
| "默认值:1.0": "Default: 1.0", | |||
| "默认值:1.25": "Default: 1.25" | |||
| ``` | |||
| ### 步骤 1.3: 验证 JSON 格式 | |||
| 确保添加的 JSON 格式正确,没有语法错误: | |||
| ```bash | |||
| cd web/src/i18n/locales | |||
| node -e "console.log('zh-CN:', JSON.parse(require('fs').readFileSync('zh-CN.json', 'utf8'))); console.log('en:', JSON.parse(require('fs').readFileSync('en.json', 'utf8')))" | |||
| ``` | |||
| 预期输出:无错误,输出两个对象的内容 | |||
| ### 步骤 1.4: 提交国际化更改 | |||
| ```bash | |||
| git add web/src/i18n/locales/zh-CN.json web/src/i18n/locales/en.json | |||
| git commit -m "feat(i18n): add advanced ratio translations | |||
| - Add Chinese translations for 5 advanced ratio fields | |||
| - Add English translations for 5 advanced ratio fields | |||
| Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>" | |||
| ``` | |||
| --- | |||
| ## Task 2: 扩展数据初始化逻辑 | |||
| **文件:** | |||
| - Modify: `web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx:59-91` | |||
| ### 步骤 2.1: 在 useEffect 中解析高级倍率 JSON | |||
| 找到 `useEffect` 开始(约第 59 行),在 `try` 块中,在现有的 `const completionRatio = JSON.parse(props.options.CompletionRatio || '{}');` 之后添加: | |||
| ```javascript | |||
| // 新增:解析高级倍率字段 | |||
| const cacheRatio = JSON.parse(props.options.CacheRatio || '{}'); | |||
| const createCacheRatio = JSON.parse(props.options.CreateCacheRatio || '{}'); | |||
| const imageRatio = JSON.parse(props.options.ImageRatio || '{}'); | |||
| const audioRatio = JSON.parse(props.options.AudioRatio || '{}'); | |||
| const audioCompletionRatio = JSON.parse(props.options.AudioCompletionRatio || '{}'); | |||
| ``` | |||
| ### 步骤 2.2: 扩展模型名称合并逻辑 | |||
| 在同一个 `useEffect` 中,找到 `const modelNames = new Set([...])`(约第 66 行),将数组扩展为: | |||
| ```javascript | |||
| const modelNames = new Set([ | |||
| ...Object.keys(modelPrice), | |||
| ...Object.keys(modelRatio), | |||
| ...Object.keys(completionRatio), | |||
| ...Object.keys(cacheRatio), | |||
| ...Object.keys(createCacheRatio), | |||
| ...Object.keys(imageRatio), | |||
| ...Object.keys(audioRatio), | |||
| ...Object.keys(audioCompletionRatio), | |||
| ]); | |||
| ``` | |||
| ### 步骤 2.3: 扩展模型数据对象 | |||
| 在同一个 `useEffect` 中,找到 `const modelData = Array.from(modelNames).map((name) => {`(约第 72 行),在返回对象中添加新字段。找到现有的: | |||
| ```javascript | |||
| const comp = completionRatio[name] === undefined ? '' : completionRatio[name]; | |||
| ``` | |||
| 在这一行后添加: | |||
| ```javascript | |||
| const cacheR = cacheRatio[name] === undefined ? '' : cacheRatio[name]; | |||
| const cacheCreationR = createCacheRatio[name] === undefined ? '' : createCacheRatio[name]; | |||
| const imageR = imageRatio[name] === undefined ? '' : imageRatio[name]; | |||
| const audioR = audioRatio[name] === undefined ? '' : audioRatio[name]; | |||
| const audioCompR = audioCompletionRatio[name] === undefined ? '' : audioCompletionRatio[name]; | |||
| ``` | |||
| ### 步骤 2.4: 更新返回对象结构 | |||
| 继续在同一个 `map` 函数中,找到返回对象(约第 78 行),更新为: | |||
| ```javascript | |||
| return { | |||
| name, | |||
| price, | |||
| ratio, | |||
| completionRatio: comp, | |||
| cacheRatio: cacheR, | |||
| cacheCreationRatio: cacheCreationR, | |||
| imageRatio: imageR, | |||
| audioRatio: audioR, | |||
| audioCompletionRatio: audioCompR, | |||
| hasConflict: price !== '' && (ratio !== '' || comp !== ''), | |||
| }; | |||
| ``` | |||
| ### 步骤 2.5: 启动开发服务器验证数据加载 | |||
| ```bash | |||
| cd web | |||
| bun run dev | |||
| ``` | |||
| 访问:http://localhost:5173/setting/ratio → 切换到"可视化倍率设置"标签 | |||
| 打开浏览器控制台,输入: | |||
| ```javascript | |||
| // 检查 models 状态是否包含新字段 | |||
| const tableRows = document.querySelectorAll('.semi-table-row'); | |||
| console.log('Table rows count:', tableRows.length); | |||
| ``` | |||
| 预期:表格正常显示,无 JavaScript 错误 | |||
| ### 步骤 2.6: 提交数据初始化更改 | |||
| ```bash | |||
| git add web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx | |||
| git commit -m "feat(ratio): parse advanced ratio fields in data initialization | |||
| - Parse 5 new JSON fields from props.options | |||
| - Extend modelNames Set to include keys from all ratio fields | |||
| - Add 5 new fields to model data objects | |||
| Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>" | |||
| ``` | |||
| --- | |||
| ## Task 3: 添加高级比例表单区域 | |||
| **文件:** | |||
| - Modify: `web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx:616-737` | |||
| ### 步骤 3.1: 找到插入位置 | |||
| 在文件中找到 `{pricingSubMode === 'token-price' && (` 块的闭合位置(约第 736 行),应该在 `)}$ 之前,`{pricingMode === 'per-request' && (` 之前。 | |||
| ### 步骤 3.2: 插入高级比例表单区域 | |||
| 在 `{pricingSubMode === 'token-price' && (` 块的闭合 `)}` 之后,`{pricingMode === 'per-request' && (` 之前,添加: | |||
| ```jsx | |||
| <Form.Section text={t('高级比例(留空使用默认值)')}> | |||
| <Form.InputNumber | |||
| field='cacheRatio' | |||
| label={t('缓存读取倍率')} | |||
| min={0} | |||
| step={0.01} | |||
| placeholder={t('默认值:1.0')} | |||
| /> | |||
| <Form.InputNumber | |||
| field='cacheCreationRatio' | |||
| label={t('缓存创建倍率(5分钟)')} | |||
| min={0} | |||
| step={0.01} | |||
| placeholder={t('默认值:1.25')} | |||
| /> | |||
| <Form.InputNumber | |||
| field='imageRatio' | |||
| label={t('图片倍率')} | |||
| min={0} | |||
| step={0.01} | |||
| placeholder={t('默认值:1.0')} | |||
| /> | |||
| <Form.InputNumber | |||
| field='audioRatio' | |||
| label={t('音频输入倍率')} | |||
| min={0} | |||
| step={0.01} | |||
| placeholder={t('默认值:1.0')} | |||
| /> | |||
| <Form.InputNumber | |||
| field='audioCompletionRatio' | |||
| label={t('音频输出倍率')} | |||
| min={0} | |||
| step={0.01} | |||
| placeholder={t('默认值:1.0')} | |||
| /> | |||
| </Form.Section> | |||
| ``` | |||
| **注意缩进**:保持与周围代码一致的缩进(约 14-16 个空格) | |||
| ### 步骤 3.2: 验证表单渲染 | |||
| 确保开发服务器仍在运行(`bun run dev`),访问:http://localhost:5173/setting/ratio | |||
| 1. 点击任意模型的"编辑"按钮 | |||
| 2. 确认定价模式为"按量计费" | |||
| 3. 向下滚动,应该能看到"高级比例(留空使用默认值)"区域 | |||
| 4. 确认 5 个输入框都有正确的标签和 placeholder | |||
| 预期: | |||
| - 新区域显示在"价格设置方式"和"固定价格"之间 | |||
| - 所有 5 个字段可见且可交互 | |||
| - placeholder 显示正确的默认值 | |||
| ### 步骤 3.3: 测试输入验证 | |||
| 在高级比例区域的任意输入框中: | |||
| 1. 输入负数(如 `-1`),预期:无法输入或显示验证错误 | |||
| 2. 输入文字(如 `abc`),预期:无法输入(InputNumber 组件自动限制) | |||
| 3. 输入小数(如 `1.23`),预期:正常接受 | |||
| 4. 输入多位小数(如 `1.23456`),预期:step=0.01 限制 | |||
| ### 步骤 3.4: 提交表单 UI 更改 | |||
| ```bash | |||
| git add web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx | |||
| git commit -m "feat(ui): add advanced ratios form section | |||
| - Add Form.Section with 5 advanced ratio input fields | |||
| - Configure min={0}, step={0.01} for all fields | |||
| - Add placeholders showing default values | |||
| - Only show when pricingMode === 'per-token' | |||
| Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>" | |||
| ``` | |||
| --- | |||
| ## Task 4: 更新编辑模型逻辑 | |||
| **文件:** | |||
| - Modify: `web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx:395-454` | |||
| ### 步骤 4.1: 在 editModel 中设置表单值 | |||
| 找到 `editModel` 函数中的 `setTimeout` 块(约第 435 行),在 `formRef.current.setValues({` 调用中添加新字段。 | |||
| 找到现有的: | |||
| ```javascript | |||
| completionTokenPrice: modelCopy.completionTokenPrice | |||
| ``` | |||
| 在这一行后添加(注意在 `})` 之前添加逗号): | |||
| ```javascript | |||
| , | |||
| // 新增:高级倍率字段 | |||
| cacheRatio: modelCopy.cacheRatio, | |||
| cacheCreationRatio: modelCopy.cacheCreationRatio, | |||
| imageRatio: modelCopy.imageRatio, | |||
| audioRatio: modelCopy.audioRatio, | |||
| audioCompletionRatio: modelCopy.audioCompletionRatio | |||
| ``` | |||
| ### 步骤 4.2: 验证编辑回显 | |||
| 1. 点击已有数据的模型(如 gpt-4)的"编辑"按钮 | |||
| 2. 在浏览器控制台中执行: | |||
| ```javascript | |||
| // 检查表单值是否正确设置 | |||
| const formApi = document.querySelector('[data-form-id]')?.__semi_form__; | |||
| if (formApi) { | |||
| console.log('Form values:', formApi.getValues()); | |||
| } | |||
| ``` | |||
| 或者更简单:直接观察高级比例区域的输入框,如果有数据应该显示对应的值 | |||
| 预期: | |||
| - 如果模型有高级倍率数据,输入框应显示对应值 | |||
| - 如果没有数据,输入框应为空 | |||
| ### 步骤 4.3: 提交编辑逻辑更改 | |||
| ```bash | |||
| git add web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx | |||
| git commit -m "feat(edit): populate advanced ratios in edit modal | |||
| - Add 5 advanced ratio fields to formRef.current.setValues | |||
| - Ensure existing ratio data displays correctly in edit modal | |||
| Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>" | |||
| ``` | |||
| --- | |||
| ## Task 5: 更新添加/更新模型逻辑 | |||
| **文件:** | |||
| - Modify: `web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx:334-382` | |||
| ### 步骤 5.1: 在 addOrUpdateModel 中处理高级倍率字段 | |||
| 找到 `addOrUpdateModel` 函数(约第 334 行),在 `const updatedModel = {` 对象中添加新字段。 | |||
| 找到现有的: | |||
| ```javascript | |||
| completionRatio: values.completionRatio || '', | |||
| ``` | |||
| 在这一行后添加: | |||
| ```javascript | |||
| // 新增:高级倍率字段 | |||
| cacheRatio: values.cacheRatio || '', | |||
| cacheCreationRatio: values.cacheCreationRatio || '', | |||
| imageRatio: values.imageRatio || '', | |||
| audioRatio: values.audioRatio || '', | |||
| audioCompletionRatio: values.audioCompletionRatio || '', | |||
| ``` | |||
| ### 步骤 5.2: 测试添加新模型 | |||
| 1. 点击"添加模型"按钮 | |||
| 2. 输入模型名称:`test-model-advanced` | |||
| 3. 在高级比例区域输入测试值: | |||
| - 缓存读取倍率:`0.5` | |||
| - 缓存创建倍率:`1.5` | |||
| - 图片倍率:`2.0` | |||
| - 音频输入倍率:`1.2` | |||
| - 音频输出倍率:`0.8` | |||
| 4. 点击"确定" | |||
| 预期: | |||
| - 模型成功添加到表格 | |||
| - 浏览器控制台无错误 | |||
| ### 步骤 5.3: 验证数据保存到状态 | |||
| 在浏览器控制台中检查: | |||
| ```javascript | |||
| // 检查是否能在表格中看到新模型 | |||
| const rows = document.querySelectorAll('.semi-table-row'); | |||
| console.log('Total rows:', rows.length); | |||
| // 应该比之前多 1 行 | |||
| ``` | |||
| ### 步骤 5.4: 提交添加/更新逻辑更改 | |||
| ```bash | |||
| git add web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx | |||
| git commit -m "feat(add): handle advanced ratios in addOrUpdateModel | |||
| - Add 5 advanced ratio fields to updatedModel object | |||
| - Use empty string fallback for undefined values | |||
| Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>" | |||
| ``` | |||
| --- | |||
| ## Task 6: 更新保存逻辑 | |||
| **文件:** | |||
| - Modify: `web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx:110-177` | |||
| ### 步骤 6.1: 扩展 output 对象 | |||
| 找到 `SubmitData` 函数中的 `const output = {`(约第 112 行),在对象中添加新字段。 | |||
| 找到现有的: | |||
| ```javascript | |||
| CompletionRatio: {}, | |||
| ``` | |||
| 在这一行后添加: | |||
| ```javascript | |||
| // 新增:高级倍率字段 | |||
| CacheRatio: {}, | |||
| CreateCacheRatio: {}, | |||
| ImageRatio: {}, | |||
| AudioRatio: {}, | |||
| AudioCompletionRatio: {} | |||
| ``` | |||
| ### 步骤 6.2: 添加数据转换逻辑 | |||
| 在同一个 `SubmitData` 函数中,找到 `models.forEach((model) => {` 块(约第 119 行),在现有的转换逻辑后添加。 | |||
| 找到: | |||
| ```javascript | |||
| if (model.completionRatio !== '') | |||
| output.CompletionRatio[model.name] = parseFloat(model.completionRatio); | |||
| ``` | |||
| 在这一行后添加: | |||
| ```javascript | |||
| // 新增:处理高级倍率字段 | |||
| if (model.cacheRatio !== '') | |||
| output.CacheRatio[model.name] = parseFloat(model.cacheRatio); | |||
| if (model.cacheCreationRatio !== '') | |||
| output.CreateCacheRatio[model.name] = parseFloat(model.cacheCreationRatio); | |||
| if (model.imageRatio !== '') | |||
| output.ImageRatio[model.name] = parseFloat(model.imageRatio); | |||
| if (model.audioRatio !== '') | |||
| output.AudioRatio[model.name] = parseFloat(model.audioRatio); | |||
| if (model.audioCompletionRatio !== '') | |||
| output.AudioCompletionRatio[model.name] = parseFloat(model.audioCompletionRatio); | |||
| ``` | |||
| ### 步骤 6.3: 扩展 finalOutput 对象 | |||
| 找到 `const finalOutput = {`(约第 137 行),在对象中添加新字段。 | |||
| 找到现有的: | |||
| ```javascript | |||
| CompletionRatio: JSON.stringify(output.CompletionRatio, null, 2), | |||
| ``` | |||
| 在这一行后添加: | |||
| ```javascript | |||
| CacheRatio: JSON.stringify(output.CacheRatio, null, 2), | |||
| CreateCacheRatio: JSON.stringify(output.CreateCacheRatio, null, 2), | |||
| ImageRatio: JSON.stringify(output.ImageRatio, null, 2), | |||
| AudioRatio: JSON.stringify(output.AudioRatio, null, 2), | |||
| AudioCompletionRatio: JSON.stringify(output.AudioCompletionRatio, null, 2), | |||
| ``` | |||
| ### 步骤 6.4: 测试保存功能 | |||
| 1. 确保 Task 5.2 中添加的 `test-model-advanced` 还在表格中 | |||
| 2. 修改它的一些高级倍率值(如将缓存读取倍率改为 `0.8`) | |||
| 3. 点击"应用更改"按钮 | |||
| 4. 打开浏览器网络标签,筛选 `/api/option/` 请求 | |||
| 预期: | |||
| - 看到多个 `PUT /api/option/` 请求 | |||
| - 其中包含 `CacheRatio`、`CreateCacheRatio` 等新增字段 | |||
| - 请求体包含 JSON 数据,键为 `test-model-advanced` | |||
| ### 步骤 6.5: 验证 API 请求体 | |||
| 在网络标签中点击任意一个 `/api/option/` 请求,查看"Payload"或"请求体": | |||
| ```json | |||
| { | |||
| "key": "CacheRatio", | |||
| "value": "{\n \"test-model-advanced\": 0.8\n}" | |||
| } | |||
| ``` | |||
| 预期: | |||
| - `value` 是格式化的 JSON 字符串(有缩进) | |||
| - 包含正确的模型名和倍率值 | |||
| ### 步骤 6.6: 提交保存逻辑更改 | |||
| ```bash | |||
| git add web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx | |||
| git commit -m "feat(save): serialize and save advanced ratio fields | |||
| - Add 5 new fields to output object | |||
| - Convert model data to float and populate output fields | |||
| - Serialize all fields to JSON in finalOutput | |||
| - Submit 7 total option fields (2 existing + 5 new) | |||
| Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>" | |||
| ``` | |||
| --- | |||
| ## Task 7: 更新输入验证逻辑 | |||
| **文件:** | |||
| - Modify: `web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx:255-270` | |||
| ### 步骤 7.1: 确认 updateModel 函数 | |||
| 找到 `updateModel` 函数(约第 255 行),确认现有的验证逻辑: | |||
| ```javascript | |||
| const updateModel = (name, field, value) => { | |||
| if (isNaN(value) && value !== '') { | |||
| showError(t('请输入数字')); | |||
| return; | |||
| } | |||
| // ... 更新逻辑 | |||
| }; | |||
| ``` | |||
| **说明**:由于我们使用 `Form.InputNumber` 组件,它已经内置了数字验证,所以 `updateModel` 函数不需要修改。`isNaN` 检查已经足够处理高级倍率字段。 | |||
| ### 步骤 7.2: 测试验证逻辑 | |||
| 在开发服务器运行的情况下: | |||
| 1. 尝试在表格中直接点击模型倍率或补全倍率输入框 | |||
| 2. 输入文字(如 `abc`) | |||
| 3. 观察是否显示错误提示"请输入数字" | |||
| **注意**:由于我们现在使用 `Form.InputNumber` 而非 `Input`,高级比例字段在弹窗中,`updateModel` 函数不会被它们调用。`Form.InputNumber` 会自动限制只能输入数字。 | |||
| ### 步骤 7.3: 验证空值处理 | |||
| 测试空值是否正确处理: | |||
| 1. 编辑一个模型 | |||
| 2. 在高级比例区域,将所有输入框留空 | |||
| 3. 点击"确定" | |||
| 4. 点击"应用更改" | |||
| 预期: | |||
| - 保存成功,无错误 | |||
| - 空值保存为空字符串,后端使用默认值 | |||
| ### 步骤 7.4: 提交验证相关更改 | |||
| 如果本任务没有代码修改,跳过提交。否则: | |||
| ```bash | |||
| git add web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx | |||
| git commit -m "feat(validation): enhance input validation for advanced ratios | |||
| Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>" | |||
| ``` | |||
| --- | |||
| ## Task 8: 完整功能测试 | |||
| **文件:** | |||
| - Test: `web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx` | |||
| ### 步骤 8.1: 测试数据初始化 | |||
| 1. 清空浏览器缓存或使用隐私模式 | |||
| 2. 访问:http://localhost:5173/setting/ratio | |||
| 3. 切换到"可视化倍率设置"标签 | |||
| 预期: | |||
| - 表格正常显示 | |||
| - 无 JavaScript 错误 | |||
| - 表格包含所有现有模型 | |||
| ### 步骤 8.2: 测试添加带高级倍率的模型 | |||
| 1. 点击"添加模型" | |||
| 2. 输入: | |||
| - 模型名称:`claude-3-5-sonnet-20241022` | |||
| - 定价模式:按量计费 | |||
| - 价格设置方式:按倍率设置 | |||
| - 模型倍率:`1.5` | |||
| - 补全倍率:`1.2` | |||
| - 缓存读取倍率:`0.1` | |||
| - 缓存创建倍率:`1.25` | |||
| - 图片倍率:留空 | |||
| - 音频输入倍率:留空 | |||
| - 音频输出倍率:留空 | |||
| 3. 点击"确定" | |||
| 预期: | |||
| - 模型成功添加 | |||
| - 表格中显示新模型 | |||
| - 无错误提示 | |||
| ### 步骤 8.3: 测试编辑高级倍率 | |||
| 1. 点击刚才添加的模型的"编辑"按钮 | |||
| 2. 修改高级倍率值: | |||
| - 缓存读取倍率:`0.15` | |||
| - 缓存创建倍率:`1.3` | |||
| - 图片倍率:`2.5` | |||
| - 音频输入倍率:`10.0` | |||
| - 音频输出倍率:`20.0` | |||
| 3. 点击"确定" | |||
| 4. 点击"应用更改" | |||
| 预期: | |||
| - 编辑成功 | |||
| - 显示"保存成功"提示 | |||
| ### 步骤 8.4: 验证数据持久化 | |||
| 刷新浏览器页面,重新访问可视化倍率设置: | |||
| 1. 点击刚才编辑的模型的"编辑"按钮 | |||
| 2. 检查高级比例区域的值是否与步骤 8.3 中设置的一致 | |||
| 预期: | |||
| - 缓存读取倍率:显示 `0.15` | |||
| - 缓存创建倍率:显示 `1.3` | |||
| - 图片倍率:显示 `2.5` | |||
| - 音频输入倍率:显示 `10.0` | |||
| - 音频输出倍率:显示 `20.0` | |||
| ### 步骤 8.5: 测试删除模型 | |||
| 1. 选择测试模型 | |||
| 2. 点击"删除"按钮 | |||
| 3. 确认删除 | |||
| 预期: | |||
| - 模型从表格中移除 | |||
| - 删除的高级倍率数据在保存时被清除(后续保存不会包含该模型) | |||
| ### 步骤 8.6: 测试边界情况 | |||
| **测试负数输入**: | |||
| 1. 尝试在高级比例输入框输入 `-1` | |||
| 2. 预期:`min={0}` 限制,无法输入负数 | |||
| **测试小数精度**: | |||
| 1. 输入 `0.123456` | |||
| 2. 预期:`step={0.01}` 限制,可以输入但会保留多位小数(Semi UI InputNumber 行为) | |||
| **测试空值保存**: | |||
| 1. 编辑模型,清空所有高级比例输入框 | |||
| 2. 点击"确定",然后"应用更改" | |||
| 3. 预期:保存成功,后端使用默认值 | |||
| ### 步骤 8.7: 测试国际化切换 | |||
| 1. 在系统设置中切换到英文(如果支持) | |||
| 2. 重新访问可视化倍率设置 | |||
| 3. 检查高级比例区域的标签是否显示为英文 | |||
| 预期: | |||
| - "高级比例(留空使用默认值)" → "Advanced Ratios (Leave empty for defaults)" | |||
| - "缓存读取倍率" → "Cache Read Ratio" | |||
| - 其他字段类似 | |||
| ### 步骤 8.8: 创建测试提交 | |||
| 如果所有测试通过,创建一个测试提交标记: | |||
| ```bash | |||
| git add web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx | |||
| git commit -m "test: verify advanced ratios functionality | |||
| - Test data initialization with empty options | |||
| - Test adding model with advanced ratios | |||
| - Test editing advanced ratios | |||
| - Test data persistence after page refresh | |||
| - Test model deletion | |||
| - Test edge cases (negative numbers, decimal precision, empty values) | |||
| - Test internationalization (English/Chinese) | |||
| All tests passed successfully. | |||
| Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>" | |||
| ``` | |||
| --- | |||
| ## Task 9: 代码清理和优化 | |||
| **文件:** | |||
| - Modify: `web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx` | |||
| ### 步骤 9.1: 检查代码格式化 | |||
| 运行代码格式化工具(如果项目配置了): | |||
| ```bash | |||
| cd web | |||
| bun run lint --fix | |||
| ``` | |||
| 预期: | |||
| - 无 lint 错误 | |||
| - 代码格式符合项目规范 | |||
| ### 步骤 9.2: 检查 console.log | |||
| 搜索并移除任何调试用的 `console.log`: | |||
| ```bash | |||
| cd web/src/pages/Setting/Ratio | |||
| grep -n "console.log" ModelSettingsVisualEditor.jsx | |||
| ``` | |||
| 如果找到任何 `console.log`,删除它们。 | |||
| ### 步骤 9.3: 验证组件性能 | |||
| 检查是否有不必要的重渲染: | |||
| 1. 打开浏览器 React DevTools(如果可用) | |||
| 2. 选择 "Profiler" 标签 | |||
| 3. 记录编辑模型时的性能 | |||
| 4. 检查是否有异常的重渲染 | |||
| 预期: | |||
| - 组件重渲染次数合理 | |||
| - 无明显的性能问题 | |||
| ### 步骤 9.4: 提交清理更改 | |||
| 如果有任何清理更改: | |||
| ```bash | |||
| git add web/src/pages/Setting/Ratio/ModelSettingsVisualEditor.jsx | |||
| git commit -m "refactor: code cleanup and optimization | |||
| - Remove debug console.log statements | |||
| - Format code with linter | |||
| - Verify component performance | |||
| Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>" | |||
| ``` | |||
| --- | |||
| ## 验证清单 | |||
| 在实施完成后,验证以下所有项目: | |||
| ### 功能验证 | |||
| - [ ] 可以添加带有高级倍率的模型 | |||
| - [ ] 可以编辑模型的高级倍率字段 | |||
| - [ ] 可以删除模型 | |||
| - [ ] 数据正确保存到后端 | |||
| - [ ] 刷新页面后数据正确加载 | |||
| - [ ] 空值字段使用默认值 | |||
| ### UI/UX 验证 | |||
| - [ ] 高级比例区域正确显示 | |||
| - [ ] 所有 5 个字段都有正确的标签 | |||
| - [ ] Placeholder 显示正确的默认值 | |||
| - [ ] 输入验证正常工作(负数、非数字) | |||
| - [ ] 中英文切换正常 | |||
| ### 代码质量 | |||
| - [ ] 无 TypeScript/JavaScript 错误 | |||
| - [ ] 无 lint 错误 | |||
| - [ ] 无调试 console.log | |||
| - [ ] 代码格式符合项目规范 | |||
| - [ ] 所有提交都有清晰的 commit message | |||
| ### 数据完整性 | |||
| - [ ] 前端字段名与后端 Option Key 一致 | |||
| - [ ] JSON 序列化/反序列化正确 | |||
| - [ ] 空值处理正确(空字符串,非 null) | |||
| - [ ] 数值转换正确(parseFloat) | |||
| --- | |||
| ## 实施完成后的最终检查 | |||
| ### 步骤 1: 查看所有提交 | |||
| ```bash | |||
| git log --oneline --no-merges | head -20 | |||
| ``` | |||
| 预期:看到一系列清晰的、原子性的提交,每个提交对应一个任务或功能点 | |||
| ### 步骤 2: 验证文件更改 | |||
| ```bash | |||
| git diff HEAD~10 HEAD --stat | |||
| ``` | |||
| 预期: | |||
| - `ModelSettingsVisualEditor.jsx` 有显著更改 | |||
| - `zh-CN.json` 和 `en.json` 有新增翻译 | |||
| - 无其他意外更改 | |||
| ### 步骤 3: 最终功能测试 | |||
| 重复 Task 8 中的所有测试步骤,确保一切正常。 | |||
| ### 步骤 4: 合并到主分支(如果需要) | |||
| ```bash | |||
| git checkout master | |||
| git merge <branch-name> | |||
| git push | |||
| ``` | |||
| --- | |||
| ## 故障排查 | |||
| ### 问题:表格不显示 | |||
| **症状**:访问可视化倍率设置页面,表格为空 | |||
| **排查**: | |||
| 1. 打开浏览器控制台,检查是否有 JavaScript 错误 | |||
| 2. 检查 `useEffect` 中的 JSON 解析是否成功 | |||
| 3. 验证 `props.options` 是否包含所有必需的字段 | |||
| **解决**: | |||
| - 确保后端 API 返回所有 7 个字段(ModelPrice, ModelRatio, CompletionRatio, CacheRatio, CreateCacheRatio, ImageRatio, AudioRatio, AudioCompletionRatio) | |||
| ### 问题:高级比例区域不显示 | |||
| **症状**:编辑模型时看不到高级比例输入框 | |||
| **排查**: | |||
| 1. 检查定价模式是否为"按量计费"(`pricingMode === 'per-token'`) | |||
| 2. 检查 Form.Section 的条件渲染 | |||
| 3. 验证国际化文本是否加载 | |||
| **解决**: | |||
| - 确保在"按量计费"模式下编辑 | |||
| - 检查 `t()` 函数是否返回正确的翻译文本 | |||
| ### 问题:数据保存失败 | |||
| **症状**:点击"应用更改"后显示错误 | |||
| **排查**: | |||
| 1. 打开浏览器网络标签,检查 `/api/option/` 请求 | |||
| 2. 查看请求响应状态码和消息 | |||
| 3. 验证请求数据格式是否正确 | |||
| **解决**: | |||
| - 确保所有 7 个字段都包含在 `finalOutput` 中 | |||
| - 检查 JSON 序列化是否正确 | |||
| - 验证后端 API 是否支持所有 7 个 Option Key | |||
| ### 问题:翻译文本显示为键名 | |||
| **症状**:界面显示 `"高级比例(留空使用默认值)"` 而非翻译后的文本 | |||
| **排查**: | |||
| 1. 检查 i18n 配置文件是否正确加载 | |||
| 2. 验证 JSON 格式是否正确 | |||
| 3. 检查是否有语法错误导致整个文件无法解析 | |||
| **解决**: | |||
| - 修复 JSON 语法错误 | |||
| - 重新加载页面 | |||
| - 清除浏览器缓存 | |||
| --- | |||
| ## 预计时间 | |||
| - Task 1 (国际化): 15 分钟 | |||
| - Task 2 (数据初始化): 20 分钟 | |||
| - Task 3 (表单 UI): 15 分钟 | |||
| - Task 4 (编辑逻辑): 10 分钟 | |||
| - Task 5 (添加/更新): 10 分钟 | |||
| - Task 6 (保存逻辑): 15 分钟 | |||
| - Task 7 (验证逻辑): 5 分钟 | |||
| - Task 8 (完整测试): 30 分钟 | |||
| - Task 9 (代码清理): 10 分钟 | |||
| **总计**: 约 2.5 小时 | |||
| --- | |||
| ## 参考资源 | |||
| ### 内部文档 | |||
| - 设计规范: `docs/superpowers/specs/2026-06-04-extended-visual-ratio-settings-design.md` | |||
| - 渠道价格设置实现: `feat/channel-pricing-extended:web/src/pages/Setting/Ratio/ChannelPricingView.jsx` | |||
| ### API 文档 | |||
| - Semi UI Form: https://douyinfe.github.io/semi-ui/form/form-inputnumber | |||
| - React hooks: https://react.dev/reference/react | |||
| ### 后端接口 | |||
| - GET `/api/option/` - 获取所有选项 | |||
| - PUT `/api/option/` - 更新单个选项(请求体: `{ key, value }`) | |||
| --- | |||
| **实施计划完成** | |||
| @@ -0,0 +1,98 @@ | |||
| --- | |||
| title: 完整删除渠道选择功能 | |||
| date: 2026-05-16 | |||
| status: approved | |||
| --- | |||
| # 设计方案:完整删除渠道选择功能 | |||
| ## 背景 | |||
| 渠道选择功能允许用户强制指定使用某个具体渠道,而非系统自动分发。有 3 个入口: | |||
| 1. Token 密钥格式 `sk-{key}:{channelId}` | |||
| 2. Playground 通道下拉框 | |||
| 3. 渠道列表 API (`GET /api/user/model_channels`, `GET /api/user/channels`) | |||
| 此外还有 `is_default` 默认渠道标记功能,用于 Playground 下拉框的默认选中项和定价页面的默认通道显示。 | |||
| **删除原因**:该功能增加代码复杂度且线上无用户使用。 | |||
| ## 决策 | |||
| - **方案**:一次性完整删除(方案 A) | |||
| - **is_default 数据库字段**:保留列,不执行 migration,只删除功能代码 | |||
| - **Distribute() 自动分发**:完全保留 | |||
| - **线上用户影响**:无,已确认无用户使用 `sk-xxx:channelId` 格式 | |||
| ## 后端改动 | |||
| ### Middleware 层 | |||
| **`middleware/auth.go`**: | |||
| - `parseTokenKey()` 简化为只提取 key,去掉冒号和短横线分割 channelId 的逻辑 | |||
| - `SetupContextForToken()` 删除第 412-415 行的 `specific_channel_id` context 设置 | |||
| **`middleware/distributor.go`**: | |||
| - `ModelRequest` 结构体删除 `ChannelId int` 字段 | |||
| - `Distribute()` 删除第 39-62 行的 `specific_channel_id` 分支,直接进入自动分发 | |||
| - `getModelRequest()` 删除第 377-382 行的 channel_id body/header 处理 | |||
| ### Controller 层 | |||
| - **删除整个** `controller/playground_channels.go` | |||
| - `controller/relay.go`:删除 `shouldRetry()` 和 `shouldRetryTaskRelay()` 中 `specific_channel_id` 判断 | |||
| - `controller/channel.go`:删除 `GetUserChannelsForBinding()` | |||
| - `controller/channel_pricing.go`:删除 `SetDefaultChannel()` 和 `ClearDefaultChannel()` 控制器函数 | |||
| ### Model 层 | |||
| - `model/ability.go`:删除 `GetModelChannelsForGroup()` | |||
| - `model/channel.go`:删除 `GetAllChannelsForBinding()` | |||
| - `model/channel_pricing.go`:删除 `GetDefaultChannelId()`、`SetDefaultChannel()`、`ClearDefaultChannel()`、`defaultChannelCache` 缓存及其锁和辅助函数 | |||
| - `model/pricing.go`:删除 `PricingModel.DefaultChannelName` 字段及其赋值逻辑 | |||
| - `model/channel_pricing_test.go`:删除相关测试用例 | |||
| ### 路由和常量 | |||
| - `constant/context_key.go`:删除 `ContextKeyTokenSpecificChannelId` | |||
| - `router/api-router.go`:删除 4 条路由(`model_channels`、`channels`、`/set_default`、`/default/*name`) | |||
| - `i18n/keys.go`:删除 `MsgChannelIdFormatError`、`MsgDistributorInvalidChannelId` | |||
| - `i18n/locales/*.yaml`:删除对应翻译条目 | |||
| ## 前端改动 | |||
| ### Playground 相关 | |||
| | 文件 | 操作 | | |||
| |------|------| | |||
| | `web/src/components/playground/SettingsPanel.jsx` | 删除通道选择 `<Select>` 组件 | | |||
| | `web/src/pages/Playground/index.jsx` | 删除 channels 加载、`X-Channel-Id` header、body `channel_id` | | |||
| | `web/src/constants/playground.constants.js` | 删除 `channelId: 0` | | |||
| | `web/src/hooks/playground/usePlaygroundState.js` | 删除 `channels` 状态 | | |||
| ### 其他前端 | |||
| | 文件 | 操作 | | |||
| |------|------| | |||
| | `web/src/components/table/tokens/modals/EditTokenModal.jsx` | 删除冗余 channels 加载代码 | | |||
| | `web/src/pages/Setting/Operation/SettingsChannelAffinity.jsx` | 从 `CONTEXT_KEY_PRESETS` 移除 `specific_channel_id` | | |||
| | `web/src/pages/Setting/Ratio/ChannelPricingView.jsx` | 删除默认通道列和星号按钮 | | |||
| | `web/src/components/table/model-pricing/view/card/PricingCardView.jsx` | 删除 `default_channel_name` 标签 | | |||
| | `web/src/components/table/model-pricing/modal/components/ChannelPricingCard.jsx` | 删除 isDefault 标签 | | |||
| | `web/src/helpers/api.js` | 删除 `channelPricingApi.setDefault` 和 `clearDefault` | | |||
| ## 不改动的部分 | |||
| - **数据库**:`is_default` 列保留在 `channel_pricing` 表,无 migration | |||
| - **Distribute() 自动分发**:核心路由逻辑完全保留 | |||
| - **计费系统**:不受影响 | |||
| - **渠道亲和性系统**:只删 `specific_channel_id` 预设项,系统本身正常 | |||
| ## 删除后的行为变化 | |||
| | 场景 | 变化前 | 变化后 | | |||
| |------|--------|--------| | |||
| | API 请求 | `sk-xxx:42` 强制走渠道 42 | 密钥匹配失败(key 格式变化) | | |||
| | Playground | 可选通道下拉框 | 全部走系统自动分发 | | |||
| | 请求失败重试 | 指定渠道时不重试 | 所有请求都能自动重试 | | |||
| | 定价页面 | 显示默认通道标记 | 不再显示 | | |||
| @@ -0,0 +1,139 @@ | |||
| # 登录 Widget 设计文档 | |||
| > 日期:2026-05-20 | 分支:master | 状态:已批准 | |||
| ## 概述 | |||
| 将现有前端登录表单封装为独立的 JS Widget,嵌入方通过 `<script>` 标签引入后渲染登录表单,调用现有 `/api/user/login` 接口完成认证,登录成功后通过回调返回用户数据。 | |||
| **核心原则**:零后端改动,纯前端实现。 | |||
| ## 嵌入方使用方式 | |||
| ```html | |||
| <div id="login-container"></div> | |||
| <script src="https://new-api.example.com/static/login-widget.js"></script> | |||
| <script> | |||
| const widget = NewApiLoginWidget.init({ | |||
| container: '#login-container', | |||
| apiUrl: 'https://new-api.example.com', | |||
| onSuccess: (data) => { | |||
| // data = { id, username, display_name, role, status, group } | |||
| console.log('登录成功:', data); | |||
| }, | |||
| onError: (err) => { | |||
| console.log('登录失败:', err.message); | |||
| } | |||
| }); | |||
| // 销毁 | |||
| widget.destroy(); | |||
| </script> | |||
| ``` | |||
| ## Widget API | |||
| ### `NewApiLoginWidget.init(options)` | |||
| | 参数 | 类型 | 必填 | 说明 | | |||
| |------|------|------|------| | |||
| | `container` | `string \| Element` | 是 | 挂载点,CSS 选择器或 DOM 元素 | | |||
| | `apiUrl` | `string` | 是 | new-api 服务地址 | | |||
| | `onSuccess` | `(data: UserData) => void` | 否 | 登录成功回调 | | |||
| | `onError` | `(err: { message: string }) => void` | 否 | 登录失败回调 | | |||
| | `theme` | `'light' \| 'dark'` | 否 | 主题,默认 `'light'` | | |||
| ### 返回值 | |||
| `{ destroy: () => void }` — 调用 `destroy()` 卸载 Widget 并清理 DOM。 | |||
| ### 回调数据格式 | |||
| ```typescript | |||
| // onSuccess 回调参数 | |||
| interface UserData { | |||
| id: number; | |||
| username: string; | |||
| display_name: string; | |||
| role: number; | |||
| status: number; | |||
| group: string; | |||
| } | |||
| ``` | |||
| ## 技术方案 | |||
| ### 构建方式 | |||
| 在 `web/widget/` 目录下创建独立的 Vite 入口,打包成 UMD 格式。 | |||
| ``` | |||
| web/widget/ | |||
| index.jsx # 入口:init() API,创建 Shadow DOM 并挂载 React 组件 | |||
| LoginForm.jsx # 登录表单组件(参考 web/src/components/auth/LoginForm.jsx) | |||
| styles.css # Widget 样式 | |||
| vite.config.js # UMD 打包配置 | |||
| ``` | |||
| ### 样式隔离 | |||
| 使用 Shadow DOM 隔离 Widget 样式,防止与嵌入方 CSS 冲突。 | |||
| ### API 调用 | |||
| 直接调用现有 `POST /api/user/login` 接口,跨域 CORS 已开启(`AllowAllOrigins + AllowCredentials`)。 | |||
| 请求: | |||
| ```json | |||
| POST {apiUrl}/api/user/login | |||
| { "username": "xxx", "password": "xxx" } | |||
| ``` | |||
| 响应(成功): | |||
| ```json | |||
| { | |||
| "success": true, | |||
| "data": { | |||
| "id": 1, | |||
| "username": "user", | |||
| "display_name": "User", | |||
| "role": 1, | |||
| "status": 1, | |||
| "group": "default" | |||
| } | |||
| } | |||
| ``` | |||
| 响应(失败): | |||
| ```json | |||
| { "success": false, "message": "用户名或密码错误" } | |||
| ``` | |||
| ### 打包产物 | |||
| - 输出:`web/build/static/login-widget.js` | |||
| - 格式:UMD(全局暴露 `NewApiLoginWidget`) | |||
| - CSS 内联到 JS bundle 中,通过 Shadow DOM 注入 | |||
| - React 和 ReactDOM 打包进 bundle(避免要求嵌入方提供 React) | |||
| ### 托管 | |||
| 由 new-api 后端的静态文件服务自动托管。构建流程: | |||
| 1. `cd web && bun run build` — 构建主前端 | |||
| 2. `cd web/widget && bun run build` — 构建 Widget(输出到 `web/build/static/`) | |||
| 3. 最终 `web/build/static/login-widget.js` 随 new-api 部署 | |||
| ## 改动范围 | |||
| | 位置 | 改动 | | |||
| |------|------| | |||
| | `web/widget/` | 新增目录,Widget 源码 | | |||
| | 后端 | **无改动** | | |||
| | 构建流程 | `Makefile` 或 `web/package.json` 添加 Widget 构建步骤 | | |||
| ## 范围限定(当前迭代) | |||
| - 仅支持**密码登录** | |||
| - 不处理 2FA | |||
| - 不处理 OAuth/Passkey 等其他登录方式 | |||
| - 不管理嵌入方的认证状态(通过回调交由嵌入方处理) | |||
| @@ -0,0 +1,704 @@ | |||
| # 海外已注册用户迁移到国内主站设计文档 | |||
| **日期**:2026-06-02 | |||
| **状态**:待评审 | |||
| **范围**:将一批已注册于海外站点的历史用户迁移为“国内主账号 + 海外同步从账号”模型,保留原登录方式和海外入口,不迁移历史日志,不考虑订阅迁移。 | |||
| --- | |||
| ## 背景与目标 | |||
| 当前项目已经具备基础的 `region_sync` 主从同步能力,核心方向是: | |||
| - 主站创建用户后向从站推送同步用户 | |||
| - 从站上的 `synced user` 消费额度后回写主站 | |||
| - 从站周期性拉取主站额度用于本地 `synced_quota` | |||
| 本次需求不是新增长期双主能力,而是把一批“历史上注册在海外站的用户”迁移为与现有同步用户一致的模型: | |||
| - 国内站成为唯一主数据源 | |||
| - 海外站保留原用户入口,但该用户转为 `synced user` | |||
| - 保留原密码登录能力 | |||
| - 保留原 OAuth 登录能力 | |||
| - 保留海外站入口可登录、可使用 | |||
| - 迁移用户的可用额度进入国内主账号 | |||
| - 不迁移调用日志、充值日志、普通操作日志 | |||
| 本次不考虑活动订阅用户,因为当前确认这批用户不存在订阅数据。 | |||
| --- | |||
| ## 非目标 | |||
| 本次设计明确不做以下事情: | |||
| - 不实现长期双向双主同步 | |||
| - 不共享国内和海外数据库 | |||
| - 不迁移 `topups`、`subscription_orders`、usage logs、quota logs | |||
| - 不重构现有普通登录链路 | |||
| - 不删除海外原用户 | |||
| - 不修改项目受保护标识、品牌、模块路径 | |||
| --- | |||
| ## 推荐方案 | |||
| 推荐方案为:`单独迁移界面 + 后端迁移任务内核`。 | |||
| ### 方案说明 | |||
| - 国内站新增一个仅 root 管理员可见的“用户迁移中心” | |||
| - 前端界面只负责: | |||
| - 创建迁移批次 | |||
| - 触发预检查 | |||
| - 处理冲突 | |||
| - 触发迁移执行 | |||
| - 查看结果与验证 | |||
| - 真正的数据迁移逻辑全部放在后端任务内核中 | |||
| - 海外站与国内站之间通过受鉴权保护的内部 HTTP 接口通信 | |||
| - 迁移完成后: | |||
| - 国内主账号持有最终主数据和额度 | |||
| - 海外原账号改为 `source=synced` | |||
| - 海外原账号通过 `remote_user_id` 指向国内主账号 | |||
| ### 不选择其他方案的原因 | |||
| - 纯脚本:冲突处理、审计、幂等和回滚边界过弱 | |||
| - 纯界面前端直写:迁移逻辑分散,重试与恢复困难 | |||
| - 登录代理/远程鉴权:改动登录、会话、OAuth 主链路,范围过大 | |||
| - 长期双主:后续冲突处理成本过高,不适合本项目现状 | |||
| --- | |||
| ## 核心设计原则 | |||
| ### 1. 国内唯一主账号 | |||
| 迁移完成后,国内站是该批用户唯一主账号归属地。之后额度主数据、用户主数据、后续运营动作都以国内站为准。 | |||
| ### 2. 海外原账号保留 | |||
| 不删除海外账号,不替换海外用户主键,不强制切登录入口。迁移后只是将其降级为同步从账号。 | |||
| ### 3. 冲突必须显式决策 | |||
| 对 `username`、`email`、OAuth 绑定冲突,不做隐式自动合并。允许自动推荐,但最终决策必须可审计。 | |||
| ### 4. 幂等优先 | |||
| 整个迁移链路按“可能中断、可能重试、可能部分成功”的现实场景设计,所有关键步骤必须可安全重跑。 | |||
| ### 5. 不做整批自动回滚 | |||
| 跨站点、带人工决策的迁移不适合整批一键回滚。系统只提供受限的单用户回退能力。 | |||
| --- | |||
| ## 整体架构 | |||
| 新增一个独立迁移模块,由以下部分组成: | |||
| - 国内站后端迁移服务 | |||
| - 海外站内部迁移接口 | |||
| - 国内站管理端迁移页面 | |||
| - 迁移批次表和迁移明细表 | |||
| 职责分工如下: | |||
| - 国内站迁移服务: | |||
| - 抓取海外候选用户快照 | |||
| - 执行国内侧冲突预检查 | |||
| - 记录批次与条目状态 | |||
| - 根据策略创建或合并国内主账号 | |||
| - 调用海外内部接口把原账号转换为 `synced` | |||
| - 做迁移后验证 | |||
| - 海外内部接口: | |||
| - 返回候选用户快照 | |||
| - 返回单用户最新快照 | |||
| - 执行“转换为 `synced` 用户” | |||
| - 管理端页面: | |||
| - 批次管理 | |||
| - 冲突处理 | |||
| - 迁移执行 | |||
| - 结果查看和验证 | |||
| --- | |||
| ## 数据模型 | |||
| 本次不复用 `quota_sync_logs`,新增两张迁移专用表。 | |||
| ### `user_migration_batches` | |||
| 表示一次迁移批次。 | |||
| 建议字段: | |||
| - `id` | |||
| - `name` | |||
| - `source_region` | |||
| - `target_region` | |||
| - `status` | |||
| - `draft` | |||
| - `scanned` | |||
| - `ready` | |||
| - `running` | |||
| - `completed` | |||
| - `failed` | |||
| - `cancelled` | |||
| - `total_users` | |||
| - `conflict_users` | |||
| - `migrated_users` | |||
| - `failed_users` | |||
| - `operator_id` | |||
| - `created_at` | |||
| - `updated_at` | |||
| ### `user_migration_items` | |||
| 表示一个海外用户在某个批次中的迁移决议和执行结果。 | |||
| 建议字段: | |||
| - `id` | |||
| - `batch_id` | |||
| - `source_user_id` | |||
| - `target_user_id` | |||
| - `source_username` | |||
| - `source_email` | |||
| - `source_quota` | |||
| - `conflict_flags` | |||
| - JSON 字符串 | |||
| - 记录 `username/email/oauth/aff_code` 等冲突 | |||
| - `resolution_strategy` | |||
| - `create_new` | |||
| - `merge_into_existing` | |||
| - `skip` | |||
| - `status` | |||
| - `pending_scan` | |||
| - `conflict` | |||
| - `ready` | |||
| - `migrating` | |||
| - `migrated` | |||
| - `skipped` | |||
| - `failed` | |||
| - `error_message` | |||
| - `snapshot` | |||
| - JSON 字符串 | |||
| - 保存预检查时的海外用户快照 | |||
| - `migration_token` | |||
| - 用于额度合并和幂等保护 | |||
| - `migrated_at` | |||
| - `created_at` | |||
| - `updated_at` | |||
| ### 唯一性建议 | |||
| - `user_migration_items(batch_id, source_user_id)` 唯一 | |||
| - 迁移执行时按 `migration_token` 保证不会重复做额度叠加 | |||
| --- | |||
| ## 状态机 | |||
| ### 批次状态 | |||
| - `draft`:已创建,未扫描 | |||
| - `scanned`:预检查完成,但仍可能存在冲突项 | |||
| - `ready`:所有项都已变成 `ready` 或 `skipped` | |||
| - `running`:正在执行迁移 | |||
| - `completed`:全部可迁移项执行成功 | |||
| - `failed`:存在失败项,需要人工修正后重试 | |||
| - `cancelled`:人工取消 | |||
| ### 条目状态 | |||
| - `pending_scan`:尚未预检查 | |||
| - `conflict`:发现冲突,等待人工决策 | |||
| - `ready`:策略已明确,可以执行迁移 | |||
| - `migrating`:迁移进行中 | |||
| - `migrated`:迁移成功 | |||
| - `skipped`:人工跳过 | |||
| - `failed`:执行失败,允许单项重试 | |||
| --- | |||
| ## 迁移范围 | |||
| ### 必迁数据 | |||
| 从海外站迁移以下主数据: | |||
| - `users` 主字段 | |||
| - 密码哈希 | |||
| - `display_name` | |||
| - `email` | |||
| - `status` | |||
| - `role` | |||
| - `group` | |||
| - `created_at` | |||
| - `setting` | |||
| - `aff_code` | |||
| - 内置 OAuth 字段 | |||
| - `github_id` | |||
| - `discord_id` | |||
| - `oidc_id` | |||
| - `wechat_id` | |||
| - `telegram_id` | |||
| - `linux_do_id` | |||
| - `user_oauth_bindings` | |||
| - 可用额度 | |||
| ### 不迁数据 | |||
| - `topups` | |||
| - `subscription_orders` | |||
| - `quota_sync_logs` | |||
| - usage logs | |||
| - 普通系统日志 | |||
| --- | |||
| ## 预检查设计 | |||
| ### 海外候选用户快照 | |||
| 海外站为国内站提供用户快照接口,返回: | |||
| - 用户主字段 | |||
| - 内置 OAuth 字段 | |||
| - 自定义 OAuth 绑定 | |||
| - `quota` | |||
| - `group` | |||
| - `status` | |||
| - `aff_code` | |||
| - `created_at` | |||
| - 可选 `setting` | |||
| - 可选 `display_name` | |||
| ### 国内预检查规则 | |||
| 对每个海外用户,在国内检查: | |||
| - `username` 是否已存在 | |||
| - `email` 是否已存在 | |||
| - 每个内置 OAuth ID 是否已被其他用户占用 | |||
| - 每个自定义 OAuth 绑定是否已被其他用户占用 | |||
| - `aff_code` 是否已存在 | |||
| ### 自定义 OAuth 绑定匹配规则 | |||
| 不能直接按 `provider_id` 比较,必须先通过 `custom_oauth_providers.slug` 建立海外与国内的 provider 映射,再按: | |||
| - `provider_slug` | |||
| - `provider_user_id` | |||
| 进行冲突判断。 | |||
| ### 冲突分级 | |||
| #### 硬冲突 | |||
| 必须人工处理,不能自动迁移: | |||
| - 用户名冲突且无法确认同一人 | |||
| - 邮箱冲突且无法确认同一人 | |||
| - OAuth 身份已绑定到另一个明确不同的国内用户 | |||
| - 自定义 OAuth 绑定指向多个不同国内候选目标 | |||
| #### 可合并冲突 | |||
| 可以由界面推荐合并,但仍需要人工确认: | |||
| - 同邮箱,且无其他矛盾 | |||
| - 唯一 OAuth 身份指向同一个国内用户,且无其他矛盾 | |||
| #### 软冲突 | |||
| 不阻塞迁移,可自动处理: | |||
| - `aff_code` 冲突 | |||
| - `display_name` 冲突 | |||
| ### 默认推荐策略 | |||
| - 无冲突:推荐 `create_new` | |||
| - 仅 `aff_code` 冲突:推荐 `create_new` | |||
| - 同邮箱且无其他矛盾、且目标国内用户尚未在海外存在其他 `synced` 副本:推荐 `merge_into_existing` | |||
| - 任意 OAuth 多对一、一对多关系:强制人工处理 | |||
| --- | |||
| ## 执行一致性约束 | |||
| - 预检查快照只用于冲突决策和界面展示,不作为最终额度结算依据。 | |||
| - 执行阶段必须重新拉取海外用户最新快照,并以该快照中的最终额度为准。 | |||
| - V1 版本要求在运维冻结窗口内执行批次。冻结窗口至少要覆盖待迁用户的充值、兑换码、额度转移和新请求入口,并等待已在途请求自然结束。 | |||
| - 如果执行时最新快照与预检查结论发生冲突性漂移(例如邮箱归属变化、OAuth 绑定目标变化、原本计划合并的国内账号已存在其他海外 `synced` 副本),该条目必须失败并要求重新扫描,不得静默沿用旧决议。 | |||
| --- | |||
| ## 单用户迁移执行流程 | |||
| 执行顺序不能改变,必须先构建国内主账号,再将海外原账号降级。执行阶段读取的“最新快照”必须发生在冻结窗口内。 | |||
| ### `create_new` | |||
| 1. 在冻结窗口内,从海外拉取该用户最新快照 | |||
| 2. 在国内开启事务 | |||
| 3. 通过“按哈希导入”的专用逻辑创建国内主账号 | |||
| - 不能走普通 `Insert()` | |||
| - 否则密码会再次哈希 | |||
| 4. 在同一个国内事务中完成以下动作: | |||
| - 创建国内主账号 | |||
| - 复制主字段 | |||
| - 复制内置 OAuth 字段 | |||
| - 复制 `user_oauth_bindings` | |||
| - 处理 `aff_code` 冲突 | |||
| - 回写 `user_migration_items.target_user_id` | |||
| 5. 如在执行阶段遇到新的唯一键冲突: | |||
| - 直接将该迁移项标记为失败 | |||
| - 要求重新扫描或人工修正 | |||
| - 不允许按用户名/邮箱自动认领为“上次已创建成功”的账号 | |||
| 6. 复制主字段: | |||
| - `username` | |||
| - `password` | |||
| - `email` | |||
| - `display_name` | |||
| - `status` | |||
| - `role` | |||
| - `group` | |||
| - `created_at` | |||
| - `setting` | |||
| 7. 复制内置 OAuth ID 字段 | |||
| 8. 复制 `user_oauth_bindings` | |||
| - 先通过 `slug` 将海外 `provider_id` 映射为国内 `provider_id` | |||
| 9. 处理邀请码 | |||
| - 如冲突则生成新的 `aff_code` | |||
| 10. 将国内额度设置为“执行时最新快照中的最终额度” | |||
| 11. 国内事务提交,得到 `cn_user_id` | |||
| 12. 调海外内部接口,将原海外用户转换为 `synced` | |||
| - `source = synced` | |||
| - `remote_user_id = cn_user_id` | |||
| - `synced_quota = 国内提交后的最终额度` | |||
| - `last_sync_at = now` | |||
| 13. 更新迁移项为成功 | |||
| ### `merge_into_existing` | |||
| 1. 在冻结窗口内拉取该用户最新快照 | |||
| 2. 锁定目标国内账号 | |||
| 3. 重新校验该用户仍应合并到这个目标 | |||
| - 同邮箱或唯一 OAuth 目标仍然成立 | |||
| - 目标国内账号在海外不存在其他 `synced` 副本 | |||
| 4. 补齐国内为空、海外有值的字段 | |||
| 5. 补齐国内缺失的 OAuth 绑定 | |||
| 6. 额度合并规则: | |||
| - `国内最终额度 = 国内原额度 + 执行时最新海外额度` | |||
| - 通过 `migration_token` 保证每个迁移项最多叠加一次 | |||
| 7. 提交国内事务 | |||
| 8. 调海外接口将原账号转换为 `synced` | |||
| 9. 更新迁移项为成功 | |||
| ### 为什么不删除海外原用户 | |||
| - 海外已有 session 兼容性更好 | |||
| - 海外已有 API token 兼容性更好 | |||
| - 现有 `synced user` 逻辑天然支持这种模型 | |||
| - 回退成本远低于删除账号再重建 | |||
| --- | |||
| ## 海外用户转换接口 | |||
| 不建议国内直接操作海外数据库,建议由海外站提供内部接口执行切换。 | |||
| ### 推荐接口 | |||
| - `GET /api/internal/migration/users` | |||
| - 分页返回候选用户快照 | |||
| - 仅返回 `source=local` 的海外用户 | |||
| - `GET /api/internal/migration/users/:id` | |||
| - 返回单用户最新快照 | |||
| - `GET /api/internal/migration/users/check-synced-copy?cn_user_id=:id` | |||
| - 检查某个国内用户是否已在海外存在 `synced` 副本 | |||
| - `POST /api/internal/migration/users/:id/convert-to-synced` | |||
| - 将海外原用户降级为 `synced` | |||
| ### 转换接口行为 | |||
| 接口需要做以下校验: | |||
| - 仅允许内部受信任调用 | |||
| - 用户存在 | |||
| - 用户尚未迁移,或已迁移且目标一致 | |||
| - `remote_user_id` 不为空 | |||
| - 如果用户已是 `synced` 但 `remote_user_id` 与请求不一致,返回冲突错误 | |||
| 幂等语义: | |||
| - 如果该用户已经是 `source=synced` | |||
| - 且 `remote_user_id` 与请求目标一致 | |||
| - 则直接返回成功 | |||
| --- | |||
| ## 管理端页面设计 | |||
| 新增一个 root-only 页面:`用户迁移中心`。 | |||
| 页面建议拆为四个区域: | |||
| ### 1. 批次列表 | |||
| 展示: | |||
| - 批次名 | |||
| - 状态 | |||
| - 总用户数 | |||
| - 冲突数 | |||
| - 已迁移数 | |||
| - 失败数 | |||
| - 创建人 | |||
| - 创建时间 | |||
| ### 2. 批次详情 | |||
| 展示用户级迁移项列表: | |||
| - 海外用户 ID | |||
| - 用户名 | |||
| - 邮箱 | |||
| - 海外额度 | |||
| - 冲突类型 | |||
| - 处理策略 | |||
| - 当前状态 | |||
| - 目标国内账号 | |||
| - 错误信息 | |||
| ### 3. 冲突处理面板 | |||
| 对单个冲突用户允许操作: | |||
| - `create_new` | |||
| - `merge_into_existing` | |||
| - `skip` | |||
| 并展示推荐策略及依据。 | |||
| ### 4. 执行与验证面板 | |||
| 提供以下操作: | |||
| - 预检查 | |||
| - 开始迁移 | |||
| - 重试失败项 | |||
| - 迁移后验证 | |||
| - 导出失败项 | |||
| --- | |||
| ## 后端 API 设计 | |||
| 新增一组管理端 API,不混入普通 `region_sync`。 | |||
| ### 国内管理端 API | |||
| - `POST /api/user-migrations/batches` | |||
| - 创建批次 | |||
| - `GET /api/user-migrations/batches` | |||
| - 批次列表 | |||
| - `GET /api/user-migrations/batches/:id` | |||
| - 批次详情 | |||
| - `POST /api/user-migrations/batches/:id/scan` | |||
| - 触发预检查 | |||
| - `POST /api/user-migrations/items/:id/resolve` | |||
| - 设置处理策略 | |||
| - 成功后重新统计批次状态;若所有项都已变为 `ready` 或 `skipped`,批次自动转为 `ready` | |||
| - `POST /api/user-migrations/batches/:id/execute` | |||
| - 执行迁移 | |||
| - `POST /api/user-migrations/batches/:id/retry` | |||
| - 仅重试失败项 | |||
| - `POST /api/user-migrations/batches/:id/verify` | |||
| - 执行迁移后校验 | |||
| ### 鉴权要求 | |||
| - 仅 root 管理员可访问 | |||
| - 不能开放给普通管理员 | |||
| --- | |||
| ## 幂等要求 | |||
| 幂等是本设计的核心要求。 | |||
| ### 扫描幂等 | |||
| - 同一批次重复 `scan` | |||
| - 不得重复插入相同 `source_user_id` | |||
| - 只能刷新已有条目的快照与冲突分析结果 | |||
| ### 执行幂等 | |||
| - 批次级重试时,已成功的条目必须跳过 | |||
| - `create_new` 条目如已存在 `target_user_id`,不得再次创建账号 | |||
| - `merge_into_existing` 额度叠加必须有一次性保护 | |||
| ### 海外转换幂等 | |||
| - 已是 `synced` | |||
| - 且 `remote_user_id` 一致 | |||
| - 直接返回成功 | |||
| ### OAuth 复制幂等 | |||
| - 已存在同一绑定则跳过 | |||
| - 不因重复插入报错 | |||
| ### 额度合并幂等 | |||
| 通过 `migration_token` 或同等级唯一标识保证: | |||
| - 每个迁移项对国内额度的修改最多成功一次 | |||
| --- | |||
| ## 冻结窗口设计 | |||
| 本次迁移不要求全站停机,但 V1 版本将“冻结窗口”定义为执行前的强制运维前置条件,而不是可选建议。 | |||
| 冻结范围应仅限该批次用户: | |||
| - 禁止充值 | |||
| - 禁止兑换码 | |||
| - 禁止额度转移 | |||
| - 禁止发起新请求 | |||
| - 等待已在途请求自然结束 | |||
| 执行要求: | |||
| - root 操作员在点击“开始迁移”前确认冻结窗口已经生效 | |||
| - 批次执行期间如果发现冻结条件被破坏,应立即停止后续条目并重新扫描 | |||
| 目的: | |||
| - 稳定海外最终额度 | |||
| - 避免预检查和执行之间继续发生额度变化 | |||
| --- | |||
| ## 回滚边界 | |||
| 本次设计不提供整批自动回滚,只提供有限的单用户回退能力。 | |||
| ### `create_new` 的可回退场景 | |||
| 如果国内主账号刚创建成功,但海外尚未转换为 `synced`: | |||
| - 可直接删除国内主账号 | |||
| 如果海外已经完成降级: | |||
| - 海外用户改回 `source=local` | |||
| - 清空 `remote_user_id` | |||
| - 将确认后的额度写回海外 `quota` | |||
| - 国内新账号按策略禁用或删除 | |||
| ### `merge_into_existing` 的回退限制 | |||
| 不建议自动回滚,因为: | |||
| - 额度已合并进既有国内账号 | |||
| - 自动扣减可能伤及已发生的后续业务动作 | |||
| 这类只建议人工修正。 | |||
| ### 系统提供的回退能力 | |||
| - 查看失败现场 | |||
| - 导出失败项 | |||
| - 对 `create_new` 类型执行受限回退 | |||
| --- | |||
| ## 迁移后验证清单 | |||
| 系统至少应提供以下自动校验: | |||
| - 国内目标主账号存在 | |||
| - 海外原账号已为 `source=synced` | |||
| - 海外 `remote_user_id` 指向正确国内账号 | |||
| - 国内额度与迁移决议一致 | |||
| - 海外 `synced_quota` 与国内额度一致,或在允许误差范围内一致 | |||
| - 密码登录抽样验证可用 | |||
| - OAuth 绑定抽样验证存在 | |||
| - 冲突用户未被误迁 | |||
| - 重跑执行不会重复加额度 | |||
| --- | |||
| ## 实施顺序 | |||
| 建议按以下顺序推进: | |||
| 1. 新增迁移数据模型与数据库迁移 | |||
| 2. 实现国内迁移服务内核 | |||
| 3. 实现海外内部迁移接口 | |||
| 4. 实现国内管理端 API | |||
| 5. 实现管理端迁移页面 | |||
| 6. 补充测试 | |||
| - 无冲突迁移 | |||
| - 邮箱冲突合并 | |||
| - OAuth 冲突阻断 | |||
| - 中断后重试 | |||
| - 已迁移项重复执行 | |||
| - 海外重复转换幂等 | |||
| 7. 先在测试库验证 | |||
| 8. 线上先跑 5-10 个用户的灰度批次 | |||
| 9. 观察成功率和验证结果后再执行全量 | |||
| --- | |||
| ## 关键实现注意事项 | |||
| ### 1. 密码导入必须是“导入哈希” | |||
| 当前用户创建逻辑会对密码再次哈希。迁移实现必须提供专用导入路径,直接写入海外已有哈希值。 | |||
| ### 2. 自定义 OAuth provider 不能直接按 `provider_id` 迁移 | |||
| 必须按 `slug` 对齐国内外 provider 后再复制绑定,否则绑定会错位。 | |||
| ### 3. 不能直接复用现有 `SyncUserCreate` | |||
| 现有 `region_sync` 的建用户请求字段不足,不能覆盖: | |||
| - 完整冲突处理 | |||
| - 自定义 OAuth 绑定复制 | |||
| - 哈希密码导入控制 | |||
| - 合并已有国内用户 | |||
| 本次迁移需要单独实现迁移接口和迁移服务。 | |||
| ### 4. 与现有 `synced user` 模型兼容 | |||
| 本方案完成后,海外侧行为应回归现有同步用户路径: | |||
| - 海外读取本地 `synced_quota` | |||
| - 海外消费额度走批量回写主站 | |||
| - 海外周期性从主站拉取额度 | |||
| --- | |||
| ## 结论 | |||
| 本次海外历史用户迁移的最佳路径是: | |||
| - 不做双主 | |||
| - 不做纯脚本黑盒迁移 | |||
| - 使用“单独迁移界面 + 后端幂等迁移任务内核” | |||
| - 将海外原用户保留并降级为 `synced` | |||
| - 将国内账号作为唯一主账号 | |||
| 这条路径最符合当前代码库已有的 `region_sync` 架构,改动集中、风险可控、审计清晰,且能保留用户现有登录方式和海外站访问体验。 | |||
| @@ -0,0 +1,511 @@ | |||
| # 扩展可视化倍率设置设计文档 | |||
| > **创建日期**:2026-06-04 | |||
| > **状态**:待实施 | |||
| > **相关组件**:ModelSettingsVisualEditor.jsx | |||
| ## 需求概述 | |||
| 在现有的 `ModelSettingsVisualEditor` 组件中添加高级倍率字段支持,允许用户为每个模型单独设置缓存、图片、音频相关的倍率。 | |||
| ### 当前限制 | |||
| 现有的可视化倍率设置只能设置: | |||
| - 模型倍率 | |||
| - 补全倍率 | |||
| ### 目标 | |||
| 新增以下5个高级倍率字段: | |||
| - 缓存读取倍率 | |||
| - 缓存创建倍率 | |||
| - 图片倍率 | |||
| - 音频输入倍率 | |||
| - 音频输出倍率 | |||
| ## 设计方案 | |||
| ### 界面设计 | |||
| #### 主表格(保持不变) | |||
| 表格列结构保持现有的简洁布局: | |||
| | 列名 | 说明 | | |||
| |------|------| | |||
| | 模型名称 | 显示模型名称,如有矛盾显示红色标签 | | |||
| | 模型固定价格 | 按次计费的价格输入框 | | |||
| | 模型倍率 | 按量计费的模型倍率 | | |||
| | 补全倍率 | 按量计费的补全倍率 | | |||
| | 操作 | 编辑、删除按钮 | | |||
| #### 编辑弹窗(新增"高级比例"区域) | |||
| 在现有的表单结构中,在"定价模式"和"价格设置方式"区域之后,增加一个新的表单区域: | |||
| ```jsx | |||
| {pricingMode === 'per-token' && ( | |||
| <> | |||
| {/* 现有的价格设置方式区域 */} | |||
| <Form.Section text={t('价格设置方式')}> | |||
| ... | |||
| </Form.Section> | |||
| {/* 新增:高级比例区域 */} | |||
| <Form.Section text={t('高级比例(留空使用默认值)')}> | |||
| <Form.InputNumber | |||
| field="cacheRatio" | |||
| label={t('缓存读取倍率')} | |||
| min={0} | |||
| step={0.01} | |||
| placeholder={t('默认值:1.0')} | |||
| /> | |||
| <Form.InputNumber | |||
| field="cacheCreationRatio" | |||
| label={t('缓存创建倍率(5分钟)')} | |||
| min={0} | |||
| step={0.01} | |||
| placeholder={t('默认值:1.25')} | |||
| /> | |||
| <Form.InputNumber | |||
| field="imageRatio" | |||
| label={t('图片倍率')} | |||
| min={0} | |||
| step={0.01} | |||
| placeholder={t('默认值:1.0')} | |||
| /> | |||
| <Form.InputNumber | |||
| field="audioRatio" | |||
| label={t('音频输入倍率')} | |||
| min={0} | |||
| step={0.01} | |||
| placeholder={t('默认值:1.0')} | |||
| /> | |||
| <Form.InputNumber | |||
| field="audioCompletionRatio" | |||
| label={t('音频输出倍率')} | |||
| min={0} | |||
| step={0.01} | |||
| placeholder={t('默认值:1.0')} | |||
| /> | |||
| </Form.Section> | |||
| </> | |||
| )} | |||
| ``` | |||
| **设计决策**: | |||
| - 只在"按量计费"模式下显示高级比例字段 | |||
| - 按次计费模式不需要这些倍率 | |||
| - 使用 `Form.InputNumber` 确保输入的是有效数字 | |||
| - `placeholder` 显示默认值提示 | |||
| - `step={0.01}` 允许小数点后两位精度 | |||
| ### 数据结构 | |||
| #### 内部状态扩展 | |||
| 在 `models` 状态数组中,每个模型对象增加5个字段: | |||
| ```javascript | |||
| { | |||
| name: 'gpt-4', | |||
| price: '', | |||
| ratio: '', | |||
| completionRatio: '', | |||
| cacheRatio: '', // 新增 | |||
| cacheCreationRatio: '', // 新增 | |||
| imageRatio: '', // 新增 | |||
| audioRatio: '', // 新增 | |||
| audioCompletionRatio: '', // 新增 | |||
| hasConflict: false | |||
| } | |||
| ``` | |||
| #### 数据初始化 | |||
| ```javascript | |||
| useEffect(() => { | |||
| try { | |||
| const modelPrice = JSON.parse(props.options.ModelPrice || '{}'); | |||
| const modelRatio = JSON.parse(props.options.ModelRatio || '{}'); | |||
| const completionRatio = JSON.parse(props.options.CompletionRatio || '{}'); | |||
| // 新增:解析高级倍率字段 | |||
| const cacheRatio = JSON.parse(props.options.CacheRatio || '{}'); | |||
| const createCacheRatio = JSON.parse(props.options.CreateCacheRatio || '{}'); | |||
| const imageRatio = JSON.parse(props.options.ImageRatio || '{}'); | |||
| const audioRatio = JSON.parse(props.options.AudioRatio || '{}'); | |||
| const audioCompletionRatio = JSON.parse(props.options.AudioCompletionRatio || '{}'); | |||
| // 合并所有模型名称 | |||
| const modelNames = new Set([ | |||
| ...Object.keys(modelPrice), | |||
| ...Object.keys(modelRatio), | |||
| ...Object.keys(completionRatio), | |||
| ...Object.keys(cacheRatio), | |||
| ...Object.keys(createCacheRatio), | |||
| ...Object.keys(imageRatio), | |||
| ...Object.keys(audioRatio), | |||
| ...Object.keys(audioCompletionRatio), | |||
| ]); | |||
| const modelData = Array.from(modelNames).map((name) => { | |||
| return { | |||
| name, | |||
| price: modelPrice[name] === undefined ? '' : modelPrice[name], | |||
| ratio: modelRatio[name] === undefined ? '' : modelRatio[name], | |||
| completionRatio: completionRatio[name] === undefined ? '' : completionRatio[name], | |||
| cacheRatio: cacheRatio[name] === undefined ? '' : cacheRatio[name], | |||
| cacheCreationRatio: createCacheRatio[name] === undefined ? '' : createCacheRatio[name], | |||
| imageRatio: imageRatio[name] === undefined ? '' : imageRatio[name], | |||
| audioRatio: audioRatio[name] === undefined ? '' : audioRatio[name], | |||
| audioCompletionRatio: audioCompletionRatio[name] === undefined ? '' : audioCompletionRatio[name], | |||
| hasConflict: modelPrice[name] !== '' && (modelRatio[name] !== '' || completionRatio[name] !== ''), | |||
| }; | |||
| }); | |||
| setModels(modelData); | |||
| } catch (error) { | |||
| console.error('JSON解析错误:', error); | |||
| } | |||
| }, [props.options]); | |||
| ``` | |||
| #### API 请求结构 | |||
| 保存时将5个新增字段序列化到对应的 JSON 字段: | |||
| ```javascript | |||
| const SubmitData = async () => { | |||
| setLoading(true); | |||
| const output = { | |||
| ModelPrice: {}, | |||
| ModelRatio: {}, | |||
| CompletionRatio: {}, | |||
| CacheRatio: {}, // 新增 | |||
| CreateCacheRatio: {}, // 新增 | |||
| ImageRatio: {}, // 新增 | |||
| AudioRatio: {}, // 新增 | |||
| AudioCompletionRatio: {} // 新增 | |||
| }; | |||
| try { | |||
| // 数据转换 | |||
| models.forEach((model) => { | |||
| if (model.price !== '') { | |||
| output.ModelPrice[model.name] = parseFloat(model.price); | |||
| } else { | |||
| if (model.ratio !== '') | |||
| output.ModelRatio[model.name] = parseFloat(model.ratio); | |||
| if (model.completionRatio !== '') | |||
| output.CompletionRatio[model.name] = parseFloat(model.completionRatio); | |||
| } | |||
| // 新增:处理高级倍率字段 | |||
| if (model.cacheRatio !== '') | |||
| output.CacheRatio[model.name] = parseFloat(model.cacheRatio); | |||
| if (model.cacheCreationRatio !== '') | |||
| output.CreateCacheRatio[model.name] = parseFloat(model.cacheCreationRatio); | |||
| if (model.imageRatio !== '') | |||
| output.ImageRatio[model.name] = parseFloat(model.imageRatio); | |||
| if (model.audioRatio !== '') | |||
| output.AudioRatio[model.name] = parseFloat(model.audioRatio); | |||
| if (model.audioCompletionRatio !== '') | |||
| output.AudioCompletionRatio[model.name] = parseFloat(model.audioCompletionRatio); | |||
| }); | |||
| // 准备API请求数组 | |||
| const finalOutput = { | |||
| ModelPrice: JSON.stringify(output.ModelPrice, null, 2), | |||
| ModelRatio: JSON.stringify(output.ModelRatio, null, 2), | |||
| CompletionRatio: JSON.stringify(output.CompletionRatio, null, 2), | |||
| CacheRatio: JSON.stringify(output.CacheRatio, null, 2), | |||
| CreateCacheRatio: JSON.stringify(output.CreateCacheRatio, null, 2), | |||
| ImageRatio: JSON.stringify(output.ImageRatio, null, 2), | |||
| AudioRatio: JSON.stringify(output.AudioRatio, null, 2), | |||
| AudioCompletionRatio: JSON.stringify(output.AudioCompletionRatio, null, 2), | |||
| }; | |||
| const requestQueue = Object.entries(finalOutput).map(([key, value]) => { | |||
| return API.put('/api/option/', { key, value }); | |||
| }); | |||
| // 批量处理请求 | |||
| const results = await Promise.all(requestQueue); | |||
| // 验证结果(保持现有逻辑) | |||
| // ... | |||
| } catch (error) { | |||
| // 错误处理 | |||
| } finally { | |||
| setLoading(false); | |||
| } | |||
| }; | |||
| ``` | |||
| ### 交互逻辑 | |||
| #### 1. 编辑模型 | |||
| 编辑现有模型时,需要在弹窗中正确回显高级倍率字段的值: | |||
| ```javascript | |||
| const editModel = (record) => { | |||
| setIsEditMode(true); | |||
| // 现有的定价模式设置逻辑 | |||
| // ... | |||
| // 创建模型副本 | |||
| const modelCopy = { ...record }; | |||
| // 如果有倍率数据,转换为 token price(现有逻辑) | |||
| if (record.ratio) { | |||
| modelCopy.tokenPrice = calculateTokenPriceFromRatio(parseFloat(record.ratio)).toString(); | |||
| if (record.completionRatio) { | |||
| modelCopy.completionTokenPrice = ( | |||
| parseFloat(modelCopy.tokenPrice) * parseFloat(record.completionRatio) | |||
| ).toString(); | |||
| } | |||
| } | |||
| // 设置当前模型 | |||
| setCurrentModel(modelCopy); | |||
| setVisible(true); | |||
| // 设置表单值 | |||
| setTimeout(() => { | |||
| if (formRef.current) { | |||
| formRef.current.setValues({ | |||
| name: modelCopy.name, | |||
| // 现有字段 | |||
| ratioInput: modelCopy.ratio, | |||
| completionRatioInput: modelCopy.completionRatio, | |||
| modelTokenPrice: modelCopy.tokenPrice, | |||
| completionTokenPrice: modelCopy.completionTokenPrice, | |||
| // 新增:高级倍率字段 | |||
| cacheRatio: modelCopy.cacheRatio, | |||
| cacheCreationRatio: modelCopy.cacheCreationRatio, | |||
| imageRatio: modelCopy.imageRatio, | |||
| audioRatio: modelCopy.audioRatio, | |||
| audioCompletionRatio: modelCopy.audioCompletionRatio, | |||
| }); | |||
| } | |||
| }, 0); | |||
| }; | |||
| ``` | |||
| #### 2. 添加/更新模型 | |||
| 在 `addOrUpdateModel` 函数中处理高级倍率字段: | |||
| ```javascript | |||
| const addOrUpdateModel = (values) => { | |||
| const existingModelIndex = models.findIndex( | |||
| (model) => model.name === values.name, | |||
| ); | |||
| const updatedModel = { | |||
| name: values.name, | |||
| price: values.price || '', | |||
| ratio: values.ratio || '', | |||
| completionRatio: values.completionRatio || '', | |||
| // 新增:高级倍率字段 | |||
| cacheRatio: values.cacheRatio || '', | |||
| cacheCreationRatio: values.cacheCreationRatio || '', | |||
| imageRatio: values.imageRatio || '', | |||
| audioRatio: values.audioRatio || '', | |||
| audioCompletionRatio: values.audioCompletionRatio || '', | |||
| }; | |||
| updatedModel.hasConflict = | |||
| updatedModel.price !== '' && | |||
| (updatedModel.ratio !== '' || updatedModel.completionRatio !== ''); | |||
| if (existingModelIndex >= 0) { | |||
| // 更新现有模型 | |||
| setModels((prev) => | |||
| prev.map((model, index) => | |||
| index !== existingModelIndex ? model : updatedModel | |||
| ) | |||
| ); | |||
| setVisible(false); | |||
| showSuccess(t('更新成功')); | |||
| } else { | |||
| // 添加新模型 | |||
| if (models.some((model) => model.name === values.name)) { | |||
| showError(t('模型名称已存在')); | |||
| return; | |||
| } | |||
| setModels((prev) => [updatedModel, ...prev]); | |||
| setVisible(false); | |||
| showSuccess(t('添加成功')); | |||
| } | |||
| }; | |||
| ``` | |||
| #### 3. 输入验证 | |||
| 在 `updateModel` 函数中增加对高级倍率字段的验证: | |||
| ```javascript | |||
| const updateModel = (name, field, value) => { | |||
| if (isNaN(value) && value !== '') { // 允许空字符串 | |||
| showError(t('请输入数字')); | |||
| return; | |||
| } | |||
| setModels((prev) => | |||
| prev.map((model) => { | |||
| if (model.name !== name) return model; | |||
| const updated = { ...model, [field]: value }; | |||
| updated.hasConflict = | |||
| updated.price !== '' && | |||
| (updated.ratio !== '' || updated.completionRatio !== ''); | |||
| return updated; | |||
| }), | |||
| ); | |||
| }; | |||
| ``` | |||
| ### 默认值处理策略 | |||
| **前端处理默认值**:用户输入框留空时,保存为空字符串,后端使用全局默认值。 | |||
| **默认值映射**: | |||
| - 缓存读取倍率:1.0 | |||
| - 缓存创建倍率:1.25 | |||
| - 图片倍率:1.0 | |||
| - 音频输入倍率:1.0 | |||
| - 音频输出倍率:1.0 | |||
| **实现方式**: | |||
| - 输入框使用 `placeholder` 提示默认值 | |||
| - 提交时空字符串保持为空,不转换 | |||
| - 后端处理空值时返回对应的默认值 | |||
| ### 模型能力处理 | |||
| **策略**:全部显示,用户自行判断 | |||
| - 所有5个高级倍率字段始终显示在编辑弹窗中 | |||
| - 用户根据自己使用的模型判断是否填写相应字段 | |||
| - 不在前端进行模型能力验证或字段隐藏 | |||
| **设计理由**: | |||
| - 简化实现,避免维护复杂的模型能力映射 | |||
| - 给用户更多控制权,允许为任意模型设置任意倍率 | |||
| - 与渠道价格设置的实现方式保持一致 | |||
| ## 实现要点 | |||
| ### 1. 字段映射 | |||
| 确保前端字段名与后端 `options` 表中的 key 一致: | |||
| | 前端字段名 | 后端 Option Key | 默认值 | | |||
| |-----------|----------------|--------| | |||
| | cacheRatio | CacheRatio | 1.0 | | |||
| | cacheCreationRatio | CreateCacheRatio | 1.25 | | |||
| | imageRatio | ImageRatio | 1.0 | | |||
| | audioRatio | AudioRatio | 1.0 | | |||
| | audioCompletionRatio | AudioCompletionRatio | 1.0 | | |||
| ### 2. 数值验证 | |||
| - 输入时验证为有效数字:`if (isNaN(value) && value !== '')` | |||
| - 范围验证:`min={0}` 确保非负数 | |||
| - 小数精度:`step={0.01}` 允许两位小数 | |||
| ### 3. 空值处理 | |||
| - 空字符串视为未设置,保存时保持为空字符串 | |||
| - 后端处理时如果值为空,返回对应的全局默认值 | |||
| - 前端解析 JSON 时,如果字段不存在,默认为空字符串:`cacheRatio[name] === undefined ? '' : cacheRatio[name]` | |||
| ### 4. 样式统一 | |||
| - 使用与现有字段一致的 `Form.InputNumber` 组件 | |||
| - 保持相同的 `label`、`placeholder` 样式 | |||
| - 表单区域使用 `Form.Section` 包裹,标题为"高级比例(留空使用默认值)" | |||
| ### 5. 国际化支持 | |||
| 需要为新增的文本添加翻译: | |||
| **中文 (zh-CN.json)**: | |||
| ```json | |||
| { | |||
| "高级比例(留空使用默认值)": "高级比例(留空使用默认值)", | |||
| "缓存读取倍率": "缓存读取倍率", | |||
| "缓存创建倍率(5分钟)": "缓存创建倍率(5分钟)", | |||
| "图片倍率": "图片倍率", | |||
| "音频输入倍率": "音频输入倍率", | |||
| "音频输出倍率": "音频输出倍率", | |||
| "默认值:1.0": "默认值:1.0", | |||
| "默认值:1.25": "默认值:1.25" | |||
| } | |||
| ``` | |||
| **英文 (en.json)**: | |||
| ```json | |||
| { | |||
| "高级比例(留空使用默认值)": "Advanced Ratios (Leave empty for defaults)", | |||
| "缓存读取倍率": "Cache Read Ratio", | |||
| "缓存创建倍率(5分钟)": "Cache Creation Ratio (5min)", | |||
| "图片倍率": "Image Ratio", | |||
| "音频输入倍率": "Audio Input Ratio", | |||
| "音频输出倍率": "Audio Output Ratio", | |||
| "默认值:1.0": "Default: 1.0", | |||
| "默认值:1.25": "Default: 1.25" | |||
| } | |||
| ``` | |||
| ## 测试要点 | |||
| ### 1. 数据初始化测试 | |||
| - 测试从空 options 初始化,确保所有字段为空字符串 | |||
| - 测试从已有数据初始化,确保正确解析所有倍率值 | |||
| ### 2. 编辑功能测试 | |||
| - 测试编辑现有模型,确保高级倍率字段正确回显 | |||
| - 测试添加新模型,确保所有字段可以正常输入 | |||
| - 测试删除模型,确保相关数据正确清理 | |||
| ### 3. 保存功能测试 | |||
| - 测试只填写部分高级倍率字段,确保正确保存 | |||
| - 测试所有高级倍率字段都留空,确保保存为空字符串 | |||
| - 测试填写数值超出范围,确保验证生效 | |||
| ### 4. 默认值测试 | |||
| - 测试留空字段,确保后端使用默认值 | |||
| - 测试输入 0,确保保存为数值 0 而不是默认值 | |||
| ### 5. 边界情况测试 | |||
| - 测试输入负数,确保被拒绝(min={0}) | |||
| - 测试输入非数字字符,确保显示错误提示 | |||
| - 测试输入非常多小数位,确保 step={0.01} 生效 | |||
| ## 参考实现 | |||
| 本设计参考了 `feat/channel-pricing-extended` 分支中的 `ChannelPricingView.jsx` 实现,特别是: | |||
| 1. **表单结构**:使用 `Form.Section` 包裹高级比例字段 | |||
| 2. **字段命名**:使用与后端 Option Key 一致的字段名 | |||
| 3. **默认值处理**:使用 placeholder 提示默认值,空字符串表示使用默认值 | |||
| 4. **输入验证**:使用 `Form.InputNumber` 配合 `min` 和 `step` 属性 | |||
| ## 实施检查清单 | |||
| - [ ] 修改 `ModelSettingsVisualEditor.jsx`,增加高级比例表单区域 | |||
| - [ ] 更新数据初始化逻辑,解析新增的5个 JSON 字段 | |||
| - [ ] 更新保存逻辑,序列化并提交新增的5个字段 | |||
| - [ ] 更新编辑模型逻辑,正确回显高级倍率字段 | |||
| - [ ] 更新添加/更新模型逻辑,处理高级倍率字段 | |||
| - [ ] 添加输入验证,确保输入有效数字 | |||
| - [ ] 添加国际化文本(中英文) | |||
| - [ ] 测试数据初始化、编辑、保存功能 | |||
| - [ ] 测试边界情况和默认值处理 | |||
| @@ -0,0 +1,131 @@ | |||
| # User Migration Delivery Plan | |||
| ## 背景 | |||
| 当前 `feat-overseas-user-migration` worktree 已经完成海外用户迁移主链路实现,并补齐了: | |||
| - 模型层迁移表与状态机 | |||
| - 海外内部迁移接口 | |||
| - 国内管理端迁移 API | |||
| - 迁移中心前端 | |||
| - 单元测试 / 控制器测试 / 路由测试 | |||
| - API E2E | |||
| - Playwright UI E2E | |||
| 本计划的目标不是继续扩功能,而是把当前能力收口到“可交付、可审阅、可部署”的状态。 | |||
| ## 当前已验证通过的内容 | |||
| 以下能力已经通过真实测试验证: | |||
| - `go test ./service/user_migration/... -v` | |||
| - `go test ./controller -run "Test.*Migration|Test.*Synced" -v` | |||
| - `go test ./model -run "Test.*Migration|Test.*Synced" -v` | |||
| - `go test ./router -run "TestUserMigrationRoutes" -v` | |||
| - `go test -v ./test/e2e -count=1` | |||
| - `cd web && bunx playwright test e2e/user-migration.spec.ts` | |||
| - `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/run-user-migration-e2e.ps1 -All` | |||
| 这些验证已经覆盖: | |||
| - 扫描层冲突分析 | |||
| - create_new / merge / skip | |||
| - fast path 恢复与 drift | |||
| - root-only 访问控制 | |||
| - verify 聚合结果 | |||
| - 内部迁移 API 边界 | |||
| - UI 迁移中心主流程 | |||
| ## 剩余工作目标 | |||
| 把当前 worktree 收敛为: | |||
| 1. 改动范围清晰 | |||
| 2. 非主线噪音最小 | |||
| 3. 文档完整 | |||
| 4. 提交路径明确 | |||
| ## 需要处理的剩余事项 | |||
| ### 1. 清理非主线前端脏 diff | |||
| 当前与迁移主线无强耦合、但会影响审阅质量的内容: | |||
| - `web/src/i18n/locales/en.json` | |||
| - `web/src/pages/Setting/Ratio/ModelRationNotSetEditor.jsx` | |||
| 目标: | |||
| - 只保留本轮确有必要的改动 | |||
| - 避免大面积格式化/编码噪音 | |||
| - 确保 `bun run build` 继续通过 | |||
| ### 2. 产出测试覆盖矩阵文档 | |||
| 新增一份文档,明确: | |||
| - 迁移功能需求点 | |||
| - 对应自动化测试名 | |||
| - 测试层级(unit / controller / router / api-e2e / ui-e2e) | |||
| 建议文件: | |||
| - `docs/testing/user-migration-test-matrix.md` | |||
| ### 3. 收敛提交范围 | |||
| 建议把当前 worktree 的改动按下面 3 组看待: | |||
| #### A. 核心后端与测试 | |||
| - `model/*migration*` | |||
| - `service/user_migration/*` | |||
| - `controller/internal_user_migration*` | |||
| - `controller/user_migration*` | |||
| - `router/user_migration_router_test.go` | |||
| #### B. 前端迁移中心 | |||
| - `web/src/components/table/users/migration/*` | |||
| - `web/src/hooks/users/useUserMigrationData.jsx` | |||
| - `web/src/pages/User/index.jsx` | |||
| - `web/e2e/*` | |||
| - `web/playwright.config.ts` | |||
| #### C. E2E 基础设施与文档 | |||
| - `test/e2e/*` | |||
| - `scripts/run-user-migration-e2e.ps1` | |||
| - `docs/testing/*` | |||
| ### 4. 最终交付前整体验证 | |||
| 在准备提交前至少再次执行: | |||
| ```bash | |||
| go test ./service/user_migration/... -v | |||
| go test ./controller -run "Test.*Migration|Test.*Synced" -v | |||
| go test ./model -run "Test.*Migration|Test.*Synced" -v | |||
| go test ./router -run "TestUserMigrationRoutes" -v | |||
| go test -v ./test/e2e -count=1 | |||
| cd web && bun run build | |||
| cd web && bunx playwright test e2e/user-migration.spec.ts | |||
| ``` | |||
| ## 建议执行顺序 | |||
| 1. 清理 `en.json` 和 `ModelRationNotSetEditor.jsx` | |||
| 2. 产出测试覆盖矩阵文档 | |||
| 3. 再跑一轮最终回归 | |||
| 4. 审查 `git diff --stat` | |||
| 5. 进入提交阶段 | |||
| ## 完成标准 | |||
| 只有同时满足以下条件,才算本轮迁移能力真正收尾: | |||
| - 迁移主链路代码无新增功能性缺口 | |||
| - Go 测试、API E2E、UI E2E 全绿 | |||
| - worktree 中无明显无关脏改动 | |||
| - 测试覆盖矩阵可供审阅 | |||
| - 提交范围已经可清晰切分 | |||
| @@ -0,0 +1,591 @@ | |||
| # 用户迁移 E2E 执行计划 | |||
| ## 背景 | |||
| 本计划用于为“海外历史用户迁移到国内主站”能力补齐完整的 E2E 测试体系。目标不是补几条临时脚本,而是形成一套可以本地稳定复跑、能定位失败原因、后续可接 CI 的测试资产。 | |||
| 本计划覆盖四个层面: | |||
| - 双节点真实运行环境 | |||
| - API 黑盒 E2E | |||
| - UI Smoke E2E | |||
| - 一键执行脚本、失败产物与文档 | |||
| ## 当前仓库现状 | |||
| | 项目 | 现状 | | |||
| |------|------| | |||
| | 浏览器 E2E 框架 | 仓库内暂无 Playwright/Cypress | | |||
| | 现有 E2E 风格 | 只有零散的 `test/e2e_*.go`、`test/e2e_*.py` | | |||
| | 迁移相关单测/集成测 | 已较完整,`service/user_migration/service_test.go` 已覆盖主链路 | | |||
| | 自动初始化 | 已支持 `POST /api/setup` 创建 root | | |||
| | 自动登录 | 已支持 `POST /api/user/login` | | |||
| | region_sync 配置 | 走 root `option` API 写入数据库 | | |||
| | CI 工作流 | 当前 worktree 下未见 `.github/workflows/` | | |||
| | 现成 `test/e2e/` 目录 | 尚未建立 | | |||
| ## 固定约定 | |||
| 以下约定在第一版中固定,不在实施过程中反复修改: | |||
| | 项目 | 固定值 | | |||
| |------|--------| | |||
| | CN 节点端口 | `3100` | | |||
| | OV 节点端口 | `3101` | | |||
| | CN 数据库 | `test-artifacts/user-migration-e2e/cn.db` | | |||
| | OV 数据库 | `test-artifacts/user-migration-e2e/ov.db` | | |||
| | 产物目录 | `test-artifacts/user-migration-e2e/` | | |||
| | CN session 名 | `cn_session` | | |||
| | OV session 名 | `ov_session` | | |||
| | Sync API Key | `e2e-sync-key` | | |||
| | 第一版运行方式 | 双进程 + 双 SQLite,不先上 Docker | | |||
| | 第一版 UI 覆盖 | Playwright smoke,不做重业务断言 | | |||
| ## 非目标 | |||
| 本计划明确不做以下事情: | |||
| - 第一版不引入 Docker Compose | |||
| - 第一版不把所有迁移正确性断言都放到浏览器层 | |||
| - 第一版不先接 CI,再回头补本地一键执行 | |||
| - 第一版不做全站级别的通用 E2E 基建重构 | |||
| ## 交付物 | |||
| 本计划完成后,仓库中应至少新增以下内容: | |||
| - `test/e2e/` Go API 黑盒 E2E | |||
| - `test/e2e/helpers/` 编排、登录、配置、seed、断言 helper | |||
| - `web/e2e/` Playwright UI smoke 测试 | |||
| - `web/playwright.config.ts` | |||
| - `scripts/run-user-migration-e2e.ps1` | |||
| - `test-artifacts/user-migration-e2e/.gitignore` | |||
| - `docs/testing/user-migration-e2e.md` | |||
| ## 总体执行顺序 | |||
| 严格按以下顺序推进,避免返工: | |||
| 1. 建目录与产物骨架 | |||
| 2. 做双节点进程编排 | |||
| 3. 做 HTTP / root setup / login / option 配置 helper | |||
| 4. 做 seed 和断言 helper | |||
| 5. 做 API E2E 主链路 | |||
| 6. 做 API E2E 边界场景 | |||
| 7. 再上 Playwright 基础设施 | |||
| 8. 做 UI smoke | |||
| 9. 做一键执行脚本 | |||
| 10. 做文档 | |||
| 11. 做全量回归验证 | |||
| 12. 视情况接 CI | |||
| ## 阶段 0:冻结基础决策 | |||
| **目标**:锁定实现边界,避免中途改架构。 | |||
| **任务** | |||
| - [ ] 确认 API E2E 使用 Go | |||
| - [ ] 确认 UI E2E 使用 Playwright | |||
| - [ ] 确认第一版仅做 UI smoke | |||
| - [ ] 确认双节点端口、数据库文件路径、session 名称 | |||
| - [ ] 确认第一版不做 Docker 化 | |||
| **验收** | |||
| - 后续阶段不再调整端口、数据库路径、session 命名 | |||
| ## 阶段 1:目录与文档骨架 | |||
| **目标**:先把结构搭起来。 | |||
| **文件** | |||
| - Create: `test/e2e/` | |||
| - Create: `test/e2e/helpers/` | |||
| - Create: `web/e2e/` | |||
| - Create: `test-artifacts/user-migration-e2e/.gitignore` | |||
| - Create: `docs/testing/user-migration-e2e.md` | |||
| **任务** | |||
| - [ ] 建立 `test/e2e/` 和 `test/e2e/helpers/` | |||
| - [ ] 建立 `web/e2e/` | |||
| - [ ] 建立 `test-artifacts/user-migration-e2e/` | |||
| - [ ] 写入产物目录 `.gitignore` | |||
| - [ ] 创建 E2E 文档骨架 | |||
| **验收** | |||
| - 所有目录和骨架文件已存在 | |||
| **建议提交** | |||
| - `chore(e2e): scaffold user migration e2e directories` | |||
| ## 阶段 2:双节点进程编排 | |||
| **目标**:自动拉起 `cn` 和 `ov` 两个真实后端实例。 | |||
| **文件** | |||
| - Create: `test/e2e/helpers/cluster.go` | |||
| **任务** | |||
| - [ ] 定义 `NodeProcess` | |||
| - [ ] 定义 `Cluster` | |||
| - [ ] 实现 `StartNode` | |||
| - [ ] 实现 `StartCluster` | |||
| - [ ] 实现 `WaitReady` | |||
| - [ ] 实现 `Stop` | |||
| - [ ] 将 stdout/stderr 重定向到 `cn.log`、`ov.log` | |||
| **强制环境变量** | |||
| - `PORT` | |||
| - `SQLITE_PATH` | |||
| - `SESSION_SECRET` | |||
| - `SESSION_NAME` | |||
| - `SESSION_SECURE=false` | |||
| - `SESSION_SAMESITE=lax` | |||
| - `GLOBAL_API_RATE_LIMIT_ENABLE=false` | |||
| - `GLOBAL_WEB_RATE_LIMIT_ENABLE=false` | |||
| - `CRITICAL_RATE_LIMIT_ENABLE=false` | |||
| - `GIN_MODE=release` | |||
| **节点差异** | |||
| - CN:`NODE_TYPE=master` | |||
| - OV:`NODE_TYPE=slave` | |||
| **验收** | |||
| - `3100` 和 `3101` 均能返回 `/api/status` | |||
| - 测试退出时双进程都能被回收 | |||
| **建议提交** | |||
| - `test(e2e): add dual-node cluster runner for migration tests` | |||
| ## 阶段 3:HTTP 客户端与认证 helper | |||
| **目标**:封装请求、会话、root 初始化和登录。 | |||
| **文件** | |||
| - Create: `test/e2e/helpers/http_client.go` | |||
| - Create: `test/e2e/helpers/auth.go` | |||
| **任务** | |||
| - [ ] 定义 `APIClient` | |||
| - [ ] 实现 `GetJSON` | |||
| - [ ] 实现 `PostJSON` | |||
| - [ ] 实现 `PutJSON` | |||
| - [ ] 实现 `DeleteJSON` | |||
| - [ ] 支持 cookie jar | |||
| - [ ] 失败时落请求/响应 dump | |||
| - [ ] 实现 `SetupRoot` | |||
| - [ ] 实现 `Login` | |||
| - [ ] 实现 root 登录快捷函数 | |||
| **验收** | |||
| - 可通过 `POST /api/setup` 自动创建 root | |||
| - 可通过 `POST /api/user/login` 自动拿到会话 | |||
| **建议提交** | |||
| - `test(e2e): add setup and session helpers for migration tests` | |||
| ## 阶段 4:region_sync 配置 helper | |||
| **目标**:通过 root `option` API 配好双节点联通。 | |||
| **文件** | |||
| - Create: `test/e2e/helpers/options.go` | |||
| **任务** | |||
| - [ ] 实现 `SetOption` | |||
| - [ ] 实现 `ConfigureCNRegionSync` | |||
| - [ ] 实现 `ConfigureOVRegionSync` | |||
| - [ ] 实现 `WaitOptionApplied` | |||
| **需要写入的配置** | |||
| CN 侧: | |||
| - `region_sync.enabled = true` | |||
| - `region_sync.region_id = "cn"` | |||
| - `region_sync.is_master = true` | |||
| - `region_sync.master_endpoint = "http://127.0.0.1:3100"` | |||
| - `region_sync.slave_endpoints = ["http://127.0.0.1:3101"]` | |||
| - `region_sync.sync_api_key = "e2e-sync-key"` | |||
| OV 侧: | |||
| - `region_sync.enabled = true` | |||
| - `region_sync.region_id = "ov"` | |||
| - `region_sync.is_master = false` | |||
| - `region_sync.master_endpoint = "http://127.0.0.1:3100"` | |||
| - `region_sync.slave_endpoints = []` | |||
| - `region_sync.sync_api_key = "e2e-sync-key"` | |||
| **注意** | |||
| - `slave_endpoints` 通过 JSON 字符串写入,不要直接写 Go slice 的字符串表示 | |||
| **验收** | |||
| - CN 能通过真实 HTTP 访问 OV 的内部迁移接口 | |||
| **建议提交** | |||
| - `test(e2e): add region sync configuration helpers` | |||
| ## 阶段 5:测试 seed helper | |||
| **目标**:每次测试从固定数据出发,结果稳定。 | |||
| **文件** | |||
| - Create: `test/e2e/helpers/seed.go` | |||
| **任务** | |||
| - [ ] 实现 `ResetDB` | |||
| - [ ] 实现 `OpenSQLite` | |||
| - [ ] 实现 `SeedCNData` | |||
| - [ ] 实现 `SeedOVData` | |||
| - [ ] 实现 `SeedOVSyncedCopy` | |||
| - [ ] 可选:实现 `SeedCustomOAuthData` | |||
| **固定数据集** | |||
| CN: | |||
| - `cn-merge-target` | |||
| - `cn-synced-copy-owner` | |||
| - `taken-username` | |||
| - `aff-owner` | |||
| OV: | |||
| - `ov-create` | |||
| - `ov-merge` | |||
| - `ov-hard-synced` | |||
| - `ov-username-conflict` | |||
| - `ov-aff-soft` | |||
| 额外: | |||
| - 一条 OV `source=synced` 用户,`remote_user_id` 指向 `cn-synced-copy-owner` | |||
| **验收** | |||
| - 重复跑测试时,扫描结果和执行结果稳定一致 | |||
| **建议提交** | |||
| - `test(e2e): add migration seed data for cn and ov nodes` | |||
| ## 阶段 6:断言与轮询 helper | |||
| **目标**:把复杂轮询和 DB 断言收敛到 helper,主测试保持清晰。 | |||
| **文件** | |||
| - Create: `test/e2e/helpers/assert.go` | |||
| **任务** | |||
| - [ ] 实现 `WaitBatchStatus` | |||
| - [ ] 实现 `GetBatchDetail` | |||
| - [ ] 实现 `FindItemBySourceUserID` | |||
| - [ ] 实现 `AssertCNUserExists` | |||
| - [ ] 实现 `AssertOVUserSynced` | |||
| - [ ] 实现 `AssertQuota` | |||
| - [ ] 实现 `AssertNoDuplicateQuotaGrant` | |||
| **验收** | |||
| - 主测试文件中不再出现重复的轮询和 JSON 解包样板 | |||
| **建议提交** | |||
| - `test(e2e): add assertion helpers for migration flow` | |||
| ## 阶段 7:API E2E 主测试骨架 | |||
| **目标**:建立统一生命周期和测试入口。 | |||
| **文件** | |||
| - Create: `test/e2e/user_migration_test.go` | |||
| **任务** | |||
| - [ ] 实现 `TestMain` | |||
| - [ ] 在 `TestMain` 中启动 cluster | |||
| - [ ] 在 `TestMain` 中 setup root | |||
| - [ ] 在 `TestMain` 中 login root | |||
| - [ ] 在 `TestMain` 中配置 region sync | |||
| - [ ] 为每个测试实现 DB 重建与 seed | |||
| **建议** | |||
| - 为保证稳定性,每个测试单独重建数据库并重启双节点 | |||
| **验收** | |||
| - 测试环境可完全自动初始化,无人工前置 | |||
| **建议提交** | |||
| - `test(e2e): add user migration test main lifecycle` | |||
| ## 阶段 8:API E2E P0 主链路 | |||
| **目标**:先证明迁移全链路真实可用。 | |||
| **文件** | |||
| - Modify: `test/e2e/user_migration_test.go` | |||
| **必须实现的用例** | |||
| - [ ] `TestUserMigrationE2E_FullFlow_GrayScenarios` | |||
| - [ ] `TestUserMigrationE2E_IdempotentRetry` | |||
| - [ ] `TestUserMigrationE2E_ExecutionTimeDriftFails` | |||
| **`FullFlow_GrayScenarios` 覆盖点** | |||
| - 创建批次 | |||
| - 调 `scan` | |||
| - 检查 5 类用户扫描结果 | |||
| - 冲突项人工 `resolve` | |||
| - 执行 `execute` | |||
| - 调 `verify` | |||
| - 校验 CN 用户状态 | |||
| - 校验 OV `source=synced` | |||
| - 校验 `remote_user_id` | |||
| - 校验 `synced_quota == cn.quota` | |||
| **验收命令** | |||
| - `go test -v ./test/e2e -run TestUserMigrationE2E_FullFlow_GrayScenarios -count=1` | |||
| - `go test -v ./test/e2e -run TestUserMigrationE2E_IdempotentRetry -count=1` | |||
| - `go test -v ./test/e2e -run TestUserMigrationE2E_ExecutionTimeDriftFails -count=1` | |||
| **建议提交** | |||
| - `test(e2e): add core migration flow tests` | |||
| ## 阶段 9:API E2E P1 边界与稳定性 | |||
| **目标**:补齐重扫、分页、权限、异常输入等场景。 | |||
| **文件** | |||
| - Modify: `test/e2e/user_migration_test.go` | |||
| **必须实现的用例** | |||
| - [ ] `TestUserMigrationE2E_RescanPreservesHumanDecision` | |||
| - [ ] `TestUserMigrationE2E_ScanPagination` | |||
| - [ ] `TestUserMigrationE2E_RootPermission` | |||
| - [ ] `TestUserMigrationE2E_ResolveInvalidPayload` | |||
| - [ ] `TestUserMigrationE2E_ScanConflict_WhenAlreadyScanning` | |||
| - [ ] `TestUserMigrationE2E_ExecuteConflict_WhenAlreadyRunning` | |||
| - [ ] `TestUserMigrationE2E_VerifyQuotaMismatchDetected` | |||
| - [ ] `TestUserMigrationE2E_SkipNeverConvertsRemote` | |||
| **验收命令** | |||
| - `go test -v ./test/e2e -count=1` | |||
| **建议提交** | |||
| - `test(e2e): add migration edge case coverage` | |||
| ## 阶段 10:Playwright 基础设施 | |||
| **目标**:搭 UI smoke 运行底座。 | |||
| **文件** | |||
| - Create: `web/playwright.config.ts` | |||
| - Create: `web/e2e/global-setup.ts` | |||
| - Create: `web/e2e/global-teardown.ts` | |||
| - Create: `web/e2e/fixtures/cluster.ts` | |||
| **任务** | |||
| - [ ] 安装 `@playwright/test` | |||
| - [ ] 配置 `testDir` | |||
| - [ ] 配置 `trace` | |||
| - [ ] 配置 `screenshot` | |||
| - [ ] 配置 `video` | |||
| - [ ] 在 `globalSetup` 中启动双节点 | |||
| - [ ] 在 `globalSetup` 中 setup root | |||
| - [ ] 在 `globalSetup` 中创建普通用户 | |||
| - [ ] 在 `globalSetup` 中 seed UI 测试数据 | |||
| - [ ] 生成 `root-storage.json` | |||
| - [ ] 生成 `user-storage.json` | |||
| - [ ] 在 `globalTeardown` 中回收进程 | |||
| **验收** | |||
| - `cd web && bunx playwright test --list` 可正常执行 | |||
| **建议提交** | |||
| - `test(ui): add playwright infrastructure for migration smoke tests` | |||
| ## 阶段 11:UI Smoke 测试 | |||
| **目标**:验证 root 迁移中心页面真正可用。 | |||
| **文件** | |||
| - Create: `web/e2e/user-migration.spec.ts` | |||
| - Modify: `web/src/components/table/users/migration/*`(如需补 `data-testid`) | |||
| **必须实现的用例** | |||
| - [ ] `root sees migration tab` | |||
| - [ ] `non-root cannot see migration tab` | |||
| - [ ] `can create batch` | |||
| - [ ] `scan updates status with polling` | |||
| - [ ] `can resolve conflict item` | |||
| - [ ] `execute shows freeze-window confirmation` | |||
| - [ ] `verify results table is rendered` | |||
| - [ ] `page refresh preserves data loading` | |||
| **规则** | |||
| - UI 层只断言页面行为,不做 DB 最终值断言 | |||
| - 重业务正确性仍由 API E2E 兜底 | |||
| **验收** | |||
| - `cd web && bunx playwright test` | |||
| **建议提交** | |||
| - `test(ui): add migration center smoke tests` | |||
| ## 阶段 12:一键执行脚本 | |||
| **目标**:本地单命令跑完全部 E2E。 | |||
| **文件** | |||
| - Create: `scripts/run-user-migration-e2e.ps1` | |||
| **任务** | |||
| - [ ] 支持 `-ApiOnly` | |||
| - [ ] 支持 `-UiOnly` | |||
| - [ ] 支持 `-All` | |||
| - [ ] 先清理旧产物 | |||
| - [ ] 跑 API E2E | |||
| - [ ] 跑 UI E2E | |||
| - [ ] 汇总结果 | |||
| - [ ] 失败时保留日志、DB、trace | |||
| **验收** | |||
| - 一条命令可以跑完整套测试 | |||
| **建议提交** | |||
| - `chore(e2e): add user migration e2e runner` | |||
| ## 阶段 13:文档补全 | |||
| **目标**:让别人能跑、能排查、能扩展。 | |||
| **文件** | |||
| - Modify: `docs/testing/user-migration-e2e.md` | |||
| **任务** | |||
| - [ ] 写明测试架构 | |||
| - [ ] 写明覆盖范围 | |||
| - [ ] 写明本地运行命令 | |||
| - [ ] 写明失败产物位置 | |||
| - [ ] 写明常见失败与排查方式 | |||
| - [ ] 写明如何新增迁移场景 | |||
| **验收** | |||
| - 新同事可按文档独立跑通 | |||
| **建议提交** | |||
| - `docs(e2e): document user migration end-to-end tests` | |||
| ## 阶段 14:总体验证 | |||
| **目标**:确认新增 E2E 不破坏现有代码。 | |||
| **必须执行** | |||
| - [ ] `go test ./...` | |||
| - [ ] `go test -v ./test/e2e -count=1` | |||
| - [ ] `cd web && bun run build` | |||
| - [ ] `cd web && bunx playwright test` | |||
| **验收** | |||
| - 后端单测全绿 | |||
| - API E2E 全绿 | |||
| - 前端构建通过 | |||
| - UI smoke 全绿 | |||
| ## 阶段 15:可选 CI 接入 | |||
| **目标**:将 E2E 纳入自动化执行。 | |||
| **说明** | |||
| 当前仓库中尚未看到 `.github/workflows/`,本阶段默认后置,不作为本轮硬性前置条件。 | |||
| **任务** | |||
| - [ ] 新增 API E2E job | |||
| - [ ] 新增 UI smoke job | |||
| - [ ] 上传 `test-artifacts/user-migration-e2e/` | |||
| **验收** | |||
| - CI 可独立稳定执行,失败时能下载产物 | |||
| ## 开始前仍需要的最少步骤 | |||
| 如果要立即开始实施,正式开工前只需要做这些最小动作: | |||
| - [ ] 确认第一批只做 API E2E 基础设施 | |||
| - [ ] 确认第一版进程启动方式是 `go run .` | |||
| - [ ] 确认允许在迁移前端组件中补 `data-testid` | |||
| - [ ] 确认第一版先不接 CI | |||
| 确认完后,直接从**阶段 1**开始执行,不需要额外准备。 | |||
| ## 最终完成定义 | |||
| 只有同时满足以下条件,才算该 E2E 计划完成: | |||
| - `test/e2e` API E2E 可稳定运行 | |||
| - `web/e2e` UI smoke 可稳定运行 | |||
| - 一键执行脚本可用 | |||
| - 失败产物可保留 | |||
| - 文档完整 | |||
| - `go test ./...` 通过 | |||
| - `cd web && bun run build` 通过 | |||
| @@ -0,0 +1,176 @@ | |||
| # 用户迁移 E2E | |||
| ## 范围 | |||
| 当前已落地两层 E2E: | |||
| - API 黑盒 E2E:真实双节点、真实 HTTP、真实 SQLite、真实 `region_sync` | |||
| - UI Smoke E2E:Playwright 驱动真实浏览器访问 `/console/user` 的迁移中心 | |||
| 对应代码位置: | |||
| - `test/e2e/user_migration_test.go` | |||
| - `test/e2e/helpers/` | |||
| - `test/e2e/cmd/user_migration_ui_server/main.go` | |||
| - `web/playwright.config.ts` | |||
| - `web/e2e/` | |||
| - `scripts/run-user-migration-e2e.ps1` | |||
| ## 已覆盖用例 | |||
| ### API E2E | |||
| - `TestUserMigrationE2E_FullFlow_GrayScenarios` | |||
| - `TestUserMigrationE2E_IdempotentRetry` | |||
| - `TestUserMigrationE2E_ExecutionTimeDriftFails` | |||
| - `TestUserMigrationE2E_RescanPreservesHumanDecision` | |||
| - `TestUserMigrationE2E_ScanPagination` | |||
| - `TestUserMigrationE2E_RootPermission` | |||
| - `TestUserMigrationE2E_ResolveInvalidPayload` | |||
| - `TestUserMigrationE2E_ScanConflict_WhenAlreadyScanning` | |||
| - `TestUserMigrationE2E_ExecuteConflict_WhenAlreadyRunning` | |||
| - `TestUserMigrationE2E_VerifyQuotaMismatchDetected` | |||
| - `TestUserMigrationE2E_SkipNeverConvertsRemote` | |||
| ### UI Smoke E2E | |||
| - 非 root 管理员看不到“用户迁移”页签 | |||
| - root 可以进入迁移中心并创建批次 | |||
| - 扫描状态能通过页面轮询从 `scanning` 进入 `scanned` | |||
| - 冲突项可以在页面中执行人工决策 | |||
| - 执行前会出现冻结窗口确认 | |||
| - 执行完成后可以触发 verify 并看到结果表 | |||
| - 页面刷新后,批次列表和详情仍可重新加载 | |||
| ## 固定约定 | |||
| - CN 端口:`3100` | |||
| - OV 端口:`3101` | |||
| - CN session:`cn_session` | |||
| - OV session:`ov_session` | |||
| - Sync API Key:`e2e-sync-key` | |||
| - API E2E 运行目录:`test-artifacts/user-migration-e2e/runs/<test-name>/` | |||
| - UI E2E 运行目录:`test-artifacts/user-migration-e2e/runs/playwright-ui/` | |||
| ## 运行方式 | |||
| 只跑 API E2E: | |||
| ```powershell | |||
| go test -v ./test/e2e -count=1 | |||
| ``` | |||
| 只跑 UI E2E: | |||
| ```powershell | |||
| cd web | |||
| bunx playwright test | |||
| ``` | |||
| 通过统一脚本执行: | |||
| ```powershell | |||
| ./scripts/run-user-migration-e2e.ps1 -ApiOnly | |||
| ./scripts/run-user-migration-e2e.ps1 -UiOnly | |||
| ./scripts/run-user-migration-e2e.ps1 -All | |||
| ``` | |||
| ## UI E2E 依赖 | |||
| Playwright 当前使用系统 Edge: | |||
| - `web/playwright.config.ts` 中固定 `channel: 'msedge'` | |||
| - 不需要额外下载 Chromium 浏览器 | |||
| 首次在新机器上跑 UI E2E,需要安装 Playwright 的 `ffmpeg`,否则失败视频无法生成: | |||
| ```powershell | |||
| cd web | |||
| bunx playwright install ffmpeg | |||
| ``` | |||
| ## UI E2E 启动流程 | |||
| `bunx playwright test` 会自动执行以下流程: | |||
| 1. `global-setup.ts` 先清理 `3100/3101` 残留监听进程 | |||
| 2. 构建前端 `web/dist` | |||
| 3. 构建 `test/e2e/cmd/user_migration_ui_server` | |||
| 4. 启动 UI helper,helper 内部调用 `helpers.PrepareEnvironment(...)` | |||
| 5. helper 完成双节点准备后写入 `test-artifacts/user-migration-e2e/playwright/server-ready.json` | |||
| 6. `global-setup.ts` 读取 ready 标记,再登录 root,并生成两套 storage state | |||
| - `root-storage.json` | |||
| - `user-storage.json`(当前实际保存的是“非 root 管理员”会话,因为 `/console/user` 受 `AdminRoute` 保护) | |||
| 7. 测试结束后由 `global-teardown.ts` 回收 UI helper 进程树 | |||
| ## 产物位置 | |||
| ### API 产物 | |||
| 每个 API 用例的产物都在: | |||
| ```text | |||
| test-artifacts/user-migration-e2e/runs/<test-name>/ | |||
| ``` | |||
| 常见文件: | |||
| - `cn.db` | |||
| - `ov.db` | |||
| - `cn.log` | |||
| - `ov.log` | |||
| - `system-logs/cn/*` | |||
| - `system-logs/ov/*` | |||
| ### UI 产物 | |||
| Playwright 相关产物在: | |||
| ```text | |||
| test-artifacts/user-migration-e2e/playwright/ | |||
| ``` | |||
| 常见文件: | |||
| - `metadata.json` | |||
| - `server-ready.json` | |||
| - `ui-server.log` | |||
| - `auth/root-storage.json` | |||
| - `auth/user-storage.json` | |||
| - `html-report/` | |||
| - `test-output/<case>/trace.zip` | |||
| - `test-output/<case>/video.webm` | |||
| - `test-output/<case>/test-failed-1.png` | |||
| ## 排查建议 | |||
| 如果 API E2E 失败,优先看: | |||
| - `test-artifacts/user-migration-e2e/runs/<test-name>/cn.log` | |||
| - `test-artifacts/user-migration-e2e/runs/<test-name>/ov.log` | |||
| - `cn.db` 中的 `user_migration_batches` | |||
| - `cn.db` 中的 `user_migration_items` | |||
| - `ov.db` 中的 `users` | |||
| 如果 UI E2E 失败,优先看: | |||
| - `test-artifacts/user-migration-e2e/playwright/ui-server.log` | |||
| - `test-artifacts/user-migration-e2e/playwright/test-output/<case>/error-context.md` | |||
| - `trace.zip` | |||
| - 失败截图和视频 | |||
| ## 如何新增迁移场景 | |||
| 新增 API 迁移场景时,优先按下面顺序扩展: | |||
| 1. 在 `test/e2e/helpers/seed.go` 里补固定 seed 或新的冲突样本 | |||
| 2. 在 `test/e2e/helpers/assert.go` 里补通用断言或轮询逻辑 | |||
| 3. 在 `test/e2e/user_migration_test.go` 里新增独立测试,尽量复用 `prepareGrayScenarioBatch` 一类的局部 helper | |||
| 4. 如果是跨节点准备逻辑变化,优先改 `test/e2e/helpers/cluster.go` 或 `options.go` | |||
| 新增 UI smoke 场景时,优先按下面顺序扩展: | |||
| 1. 先判断是否需要新的稳定选择器;需要时只在迁移中心组件上补 `data-testid` | |||
| 2. 在 `web/e2e/user-migration.spec.ts` 中新增页面行为断言,不做 DB 最终值断言 | |||
| 3. 如果需要新的登录态或前置环境,改 `web/e2e/global-setup.ts` | |||
| 4. 若新增场景改变运行产物或依赖,再同步更新本文档 | |||
| @@ -343,7 +343,24 @@ type ResponsesOutput struct { | |||
| Size string `json:"size"` | |||
| CallId string `json:"call_id,omitempty"` | |||
| Name string `json:"name,omitempty"` | |||
| Arguments string `json:"arguments,omitempty"` | |||
| // Arguments can be a JSON string or a JSON object (e.g. gpt-5.4 returns objects for some tools) | |||
| Arguments json.RawMessage `json:"arguments,omitempty"` | |||
| } | |||
| // GetArguments returns the arguments as a plain string. | |||
| // If the upstream sent a JSON string value, the inner string is returned (unescaped). | |||
| // If the upstream sent a JSON object, the raw JSON bytes are returned as a string. | |||
| func (r *ResponsesOutput) GetArguments() string { | |||
| if len(r.Arguments) == 0 { | |||
| return "" | |||
| } | |||
| if r.Arguments[0] == '"' { | |||
| var s string | |||
| if err := json.Unmarshal(r.Arguments, &s); err == nil { | |||
| return s | |||
| } | |||
| } | |||
| return string(r.Arguments) | |||
| } | |||
| type ResponsesOutputContent struct { | |||
| @@ -33,9 +33,11 @@ require ( | |||
| github.com/jinzhu/copier v0.4.0 | |||
| github.com/joho/godotenv v1.5.1 | |||
| github.com/mewkiz/flac v1.0.13 | |||
| github.com/mojocn/base64Captcha v1.3.8 | |||
| github.com/nicksnyder/go-i18n/v2 v2.6.1 | |||
| github.com/pkg/errors v0.9.1 | |||
| github.com/pquerna/otp v1.5.0 | |||
| github.com/prometheus/client_golang v1.22.0 | |||
| github.com/samber/hot v0.11.0 | |||
| github.com/samber/lo v1.52.0 | |||
| github.com/shirou/gopsutil v3.21.11+incompatible | |||
| @@ -111,6 +113,7 @@ require ( | |||
| github.com/json-iterator/go v1.1.12 // indirect | |||
| github.com/klauspost/compress v1.18.0 // indirect | |||
| github.com/klauspost/cpuid/v2 v2.3.0 // indirect | |||
| github.com/kylelemons/godebug v1.1.0 // indirect | |||
| github.com/leodido/go-urn v1.4.0 // indirect | |||
| github.com/mattn/go-isatty v0.0.20 // indirect | |||
| github.com/mewkiz/pkg v0.0.0-20250417130911-3f050ff8c56d // indirect | |||
| @@ -118,12 +121,10 @@ require ( | |||
| github.com/mitchellh/mapstructure v1.5.0 // indirect | |||
| github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect | |||
| github.com/modern-go/reflect2 v1.0.2 // indirect | |||
| github.com/mojocn/base64Captcha v1.3.8 // indirect | |||
| github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect | |||
| github.com/ncruces/go-strftime v0.1.9 // indirect | |||
| github.com/pelletier/go-toml/v2 v2.2.1 // indirect | |||
| github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect | |||
| github.com/prometheus/client_golang v1.22.0 // indirect | |||
| github.com/prometheus/client_model v0.6.1 // indirect | |||
| github.com/prometheus/common v0.62.0 // indirect | |||
| github.com/prometheus/procfs v0.15.1 // indirect | |||
| @@ -203,6 +203,8 @@ github.com/kr/pty v1.1.8/go.mod h1:O1sed60cT9XZ5uDucP5qwvh+TE3NnUj51EiZO/lmSfw= | |||
| github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= | |||
| github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= | |||
| github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= | |||
| github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= | |||
| github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= | |||
| github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII= | |||
| github.com/leodido/go-urn v1.2.1/go.mod h1:zt4jvISO2HfUBqxjfIshjdMTYS56ZS/qv49ictyFfxY= | |||
| github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ= | |||
| @@ -54,6 +54,7 @@ const ( | |||
| MsgRedemptionUsed = "redemption.used" | |||
| MsgRedemptionExpired = "redemption.expired" | |||
| MsgRedemptionFailed = "redemption.failed" | |||
| MsgRedemptionSyncedUserDenied = "redemption.synced_user_denied" | |||
| MsgRedemptionNotProvided = "redemption.not_provided" | |||
| MsgRedemptionExpireTimeInvalid = "redemption.expire_time_invalid" | |||
| ) | |||
| @@ -78,6 +79,7 @@ const ( | |||
| MsgUserNoPermissionHigherLevel = "user.no_permission_higher_level" | |||
| MsgUserCannotCreateHigherLevel = "user.cannot_create_higher_level" | |||
| MsgUserCannotDeleteRootUser = "user.cannot_delete_root_user" | |||
| MsgSyncedUserQuotaCannotModify = "user.synced_user_quota_cannot_modify" | |||
| MsgUserCannotDisableRootUser = "user.cannot_disable_root_user" | |||
| MsgUserCannotDemoteRootUser = "user.cannot_demote_root_user" | |||
| MsgUserAlreadyAdmin = "user.already_admin" | |||
| @@ -152,7 +154,6 @@ const ( | |||
| // Channel related messages | |||
| const ( | |||
| MsgChannelNotExists = "channel.not_exists" | |||
| MsgChannelIdFormatError = "channel.id_format_error" | |||
| MsgChannelNoAvailableKey = "channel.no_available_key" | |||
| MsgChannelGetListFailed = "channel.get_list_failed" | |||
| MsgChannelGetTagsFailed = "channel.get_tags_failed" | |||
| @@ -291,7 +292,6 @@ const ( | |||
| // Distributor related messages | |||
| const ( | |||
| MsgDistributorInvalidRequest = "distributor.invalid_request" | |||
| MsgDistributorInvalidChannelId = "distributor.invalid_channel_id" | |||
| MsgDistributorChannelDisabled = "distributor.channel_disabled" | |||
| MsgDistributorTokenNoModelAccess = "distributor.token_no_model_access" | |||
| MsgDistributorTokenModelForbidden = "distributor.token_model_forbidden" | |||
| @@ -46,6 +46,7 @@ redemption.invalid: "Invalid redemption code" | |||
| redemption.used: "This redemption code has been used" | |||
| redemption.expired: "This redemption code has expired" | |||
| redemption.failed: "Redemption failed, please try again later" | |||
| redemption.synced_user_denied: "Synced users cannot use redemption codes" | |||
| redemption.not_provided: "Redemption code not provided" | |||
| redemption.expire_time_invalid: "Expiration time cannot be earlier than current time" | |||
| @@ -68,6 +69,7 @@ user.no_permission_same_level: "No permission to access users of same or higher | |||
| user.no_permission_higher_level: "No permission to update users of same or higher permission level" | |||
| user.cannot_create_higher_level: "Cannot create users with permission level equal to or higher than yourself" | |||
| user.cannot_delete_root_user: "Cannot delete super administrator account" | |||
| user.synced_user_quota_cannot_modify: "Synced user quota is managed by the master node and cannot be modified" | |||
| user.cannot_disable_root_user: "Cannot disable super administrator user" | |||
| user.cannot_demote_root_user: "Cannot demote super administrator user" | |||
| user.already_admin: "This user is already an administrator" | |||
| @@ -132,7 +134,6 @@ topup.invalid_quota: "Invalid top-up quota" | |||
| # Channel messages | |||
| channel.not_exists: "Channel does not exist" | |||
| channel.id_format_error: "Channel ID format error" | |||
| channel.no_available_key: "No available channel keys" | |||
| channel.get_list_failed: "Failed to get channel list, please try again later" | |||
| channel.get_tags_failed: "Failed to get tags, please try again later" | |||
| @@ -243,7 +244,6 @@ common.invalid_input: "Invalid input" | |||
| # Distributor messages | |||
| distributor.invalid_request: "Invalid request: {{.Error}}" | |||
| distributor.invalid_channel_id: "Invalid channel ID" | |||
| distributor.channel_disabled: "This channel has been disabled" | |||
| distributor.token_no_model_access: "This token has no access to any models" | |||
| distributor.token_model_forbidden: "This token has no access to model {{.Model}}" | |||
| @@ -47,6 +47,7 @@ redemption.invalid: "无效的兑换码" | |||
| redemption.used: "该兑换码已被使用" | |||
| redemption.expired: "该兑换码已过期" | |||
| redemption.failed: "兑换失败,请稍后重试" | |||
| redemption.synced_user_denied: "同步用户不允许使用兑换码" | |||
| redemption.not_provided: "未提供兑换码" | |||
| redemption.expire_time_invalid: "过期时间不能早于当前时间" | |||
| @@ -69,6 +70,7 @@ user.no_permission_same_level: "无权获取同级或更高等级用户的信息 | |||
| user.no_permission_higher_level: "无权更新同权限等级或更高权限等级的用户信息" | |||
| user.cannot_create_higher_level: "无法创建权限大于等于自己的用户" | |||
| user.cannot_delete_root_user: "不能删除超级管理员账户" | |||
| user.synced_user_quota_cannot_modify: "同步用户余额由主节点管理,不可修改" | |||
| user.cannot_disable_root_user: "无法禁用超级管理员用户" | |||
| user.cannot_demote_root_user: "无法降级超级管理员用户" | |||
| user.already_admin: "该用户已经是管理员" | |||
| @@ -133,7 +135,6 @@ topup.invalid_quota: "无效的充值额度" | |||
| # Channel messages | |||
| channel.not_exists: "渠道不存在" | |||
| channel.id_format_error: "渠道ID格式错误" | |||
| channel.no_available_key: "没有可用的渠道密钥" | |||
| channel.get_list_failed: "获取渠道列表失败,请稍后重试" | |||
| channel.get_tags_failed: "获取标签失败,请稍后重试" | |||
| @@ -244,7 +245,6 @@ common.invalid_input: "输入不合法" | |||
| # Distributor messages | |||
| distributor.invalid_request: "无效的请求,{{.Error}}" | |||
| distributor.invalid_channel_id: "无效的渠道 Id" | |||
| distributor.channel_disabled: "该渠道已被禁用" | |||
| distributor.token_no_model_access: "该令牌无权访问任何模型" | |||
| distributor.token_model_forbidden: "该令牌无权访问模型 {{.Model}}" | |||
| @@ -47,6 +47,7 @@ redemption.invalid: "無效的兌換碼" | |||
| redemption.used: "該兌換碼已被使用" | |||
| redemption.expired: "該兌換碼已過期" | |||
| redemption.failed: "兌換失敗,請稍後重試" | |||
| redemption.synced_user_denied: "同步使用者不允許使用兌換碼" | |||
| redemption.not_provided: "未提供兌換碼" | |||
| redemption.expire_time_invalid: "過期時間不能早於當前時間" | |||
| @@ -69,6 +70,7 @@ user.no_permission_same_level: "無權獲取同級或更高等級使用者的資 | |||
| user.no_permission_higher_level: "無權更新同權限等級或更高權限等級的使用者資訊" | |||
| user.cannot_create_higher_level: "無法建立權限大於等於自己的使用者" | |||
| user.cannot_delete_root_user: "不能刪除超級管理員帳號" | |||
| user.synced_user_quota_cannot_modify: "同步使用者餘額由主節點管理,不可修改" | |||
| user.cannot_disable_root_user: "無法禁用超級管理員使用者" | |||
| user.cannot_demote_root_user: "無法降級超級管理員使用者" | |||
| user.already_admin: "該使用者已經是管理員" | |||
| @@ -133,7 +135,6 @@ topup.invalid_quota: "無效的充值額度" | |||
| # Channel messages | |||
| channel.not_exists: "管道不存在" | |||
| channel.id_format_error: "管道ID格式錯誤" | |||
| channel.no_available_key: "沒有可用的管道密鑰" | |||
| channel.get_list_failed: "獲取管道列表失敗,請稍後重試" | |||
| channel.get_tags_failed: "獲取標籤失敗,請稍後重試" | |||
| @@ -244,7 +245,6 @@ common.invalid_input: "輸入不合法" | |||
| # Distributor messages | |||
| distributor.invalid_request: "無效的請求,{{.Error}}" | |||
| distributor.invalid_channel_id: "無效的管道 Id" | |||
| distributor.channel_disabled: "該管道已被禁用" | |||
| distributor.token_no_model_access: "該令牌無權存取任何模型" | |||
| distributor.token_model_forbidden: "該令牌無權存取模型 {{.Model}}" | |||
| @@ -0,0 +1,23 @@ | |||
| { | |||
| "contents": { | |||
| "role": "USER", | |||
| "parts": [ | |||
| { | |||
| "text": "Create a Tom and Jerry Poster." | |||
| } | |||
| ] | |||
| }, | |||
| "generationConfig": { | |||
| "responseModalities": [ | |||
| "IMAGE" | |||
| ], | |||
| "imageConfig": { | |||
| "aspectRatio": "16:9" | |||
| } | |||
| }, | |||
| "safetySettings": { | |||
| "method": "PROBABILITY", | |||
| "category": "HARM_CATEGORY_DANGEROUS_CONTENT", | |||
| "threshold": "BLOCK_MEDIUM_AND_ABOVE" | |||
| } | |||
| } | |||
| @@ -0,0 +1 @@ | |||
| {"contents":[{"role":"USER","parts":[{"text":"Create a Tom and Jerry Poster."}]}],"generationConfig":{"responseModalities":["IMAGE"],"imageConfig":{"aspectRatio":"16:9"}}} | |||
| @@ -0,0 +1 @@ | |||
| {"contents":{"role":"USER","parts":[{"text":"Create a Tom and Jerry Poster."}]},"generationConfig":{"responseModalities":["IMAGE"],"imageConfig":{"aspectRatio":"16:9"}}} | |||
| @@ -0,0 +1 @@ | |||
| {"contents":{"role":"USER","parts":[{"text":"Create a Tom and Jerry Poster."}]},"generationConfig":{"responseModalities":["IMAGE"],"imageConfig":{"aspectRatio":"16:9"}}} | |||
| @@ -0,0 +1 @@ | |||
| {"model":"google/gemini-3-pro-image-preview","messages":[{"role":"user","content":"Create a Tom and Jerry Poster."}],"max_tokens":256} | |||
| @@ -183,8 +183,13 @@ func main() { | |||
| Path: "/", | |||
| MaxAge: 2592000, // 30 days | |||
| HttpOnly: true, | |||
| Secure: false, | |||
| SameSite: http.SameSiteStrictMode, | |||
| Secure: os.Getenv("SESSION_SECURE") != "false", | |||
| SameSite: func() http.SameSite { | |||
| if os.Getenv("SESSION_SAMESITE") == "strict" { | |||
| return http.SameSiteStrictMode | |||
| } | |||
| return http.SameSiteNoneMode | |||
| }(), | |||
| }) | |||
| // 支持通过环境变量自定义 session 名称,用于多节点部署时避免 cookie 冲突 | |||
| sessionName := os.Getenv("SESSION_NAME") | |||
| @@ -286,6 +291,7 @@ func InitResources() error { | |||
| // Initialize options, should after model.InitDB() | |||
| model.InitOptionMap() | |||
| model.LoadOptionsFromDatabase() | |||
| // 清理旧的磁盘缓存文件 | |||
| common.CleanupOldCacheFiles() | |||
| @@ -1,14 +1,18 @@ | |||
| FRONTEND_DIR = ./web | |||
| BACKEND_DIR = . | |||
| .PHONY: all build-frontend start-backend docker-build docker-push | |||
| .PHONY: all build-frontend build-widget start-backend docker-build docker-push | |||
| all: build-frontend start-backend | |||
| all: build-frontend build-widget start-backend | |||
| build-frontend: | |||
| @echo "Building frontend..." | |||
| @cd $(FRONTEND_DIR) && bun install && DISABLE_ESLINT_PLUGIN='true' VITE_REACT_APP_VERSION=$(cat VERSION) bun run build | |||
| build-widget: | |||
| @echo "Building login widget..." | |||
| @cd $(FRONTEND_DIR)/widget && bun install && bun run build | |||
| start-backend: | |||
| @echo "Starting backend dev server..." | |||
| @cd $(BACKEND_DIR) && go run main.go & | |||
| @@ -17,7 +21,9 @@ start-backend: | |||
| DOCKER_REGISTRY := registry.cn-hangzhou.aliyuncs.com/fengsilin/new-api | |||
| BRANCH_NAME := $(shell git rev-parse --abbrev-ref HEAD | sed 's/\//-/g') | |||
| BUILD_TIME := $(shell date +%Y%m%d%H%M) | |||
| DOCKER_TAG := $(DOCKER_REGISTRY):$(BUILD_TIME)-$(BRANCH_NAME) | |||
| COMMIT_SHORT := $(shell git rev-parse --short HEAD) | |||
| GIT_DIRTY := $(shell git diff --quiet && git diff --cached --quiet || echo "-dirty") | |||
| DOCKER_TAG := $(DOCKER_REGISTRY):$(BUILD_TIME)-$(BRANCH_NAME)-$(COMMIT_SHORT)$(GIT_DIRTY) | |||
| docker-build: | |||
| @echo "Building Docker image with tag: $(DOCKER_TAG)" | |||
| @@ -30,29 +30,12 @@ func validUserInfo(username string, role int) bool { | |||
| return true | |||
| } | |||
| // parseTokenKey 解析令牌字符串,提取 key 和可选的 channelId | |||
| // 支持格式:sk-{key} 或 sk-{key}:{channelId} 或 sk-{key}-{channelId} | |||
| // 返回:key, parts(parts[0]=key, parts[1]=channelId 如果指定) | |||
| // parseTokenKey 解析令牌字符串,提取 key | |||
| // 支持格式:sk-{key} | |||
| func parseTokenKey(key string) (string, []string) { | |||
| key = strings.TrimPrefix(key, "sk-") | |||
| // 先按冒号分割获取 channelId(新格式:sk-{key}:{channelId}) | |||
| var colonChannelId string | |||
| if colonIdx := strings.Index(key, ":"); colonIdx != -1 { | |||
| colonChannelId = key[colonIdx+1:] | |||
| key = key[:colonIdx] | |||
| } | |||
| parts := strings.Split(key, "-") | |||
| key = parts[0] | |||
| // 如果冒号方式指定了渠道,验证并追加到 parts | |||
| if colonChannelId != "" { | |||
| // 验证 channelId 为有效数字,防止 sk-abc:def:ghi 等格式 | |||
| if _, err := strconv.Atoi(colonChannelId); err == nil { | |||
| parts = append(parts, colonChannelId) | |||
| } | |||
| } | |||
| return key, parts | |||
| } | |||
| @@ -192,6 +175,20 @@ func RootAuth() func(c *gin.Context) { | |||
| } | |||
| } | |||
| func MasterNodeOnly() func(c *gin.Context) { | |||
| return func(c *gin.Context) { | |||
| if !common.IsMasterNode { | |||
| c.JSON(http.StatusForbidden, gin.H{ | |||
| "success": false, | |||
| "message": "only master node can access this endpoint", | |||
| }) | |||
| c.Abort() | |||
| return | |||
| } | |||
| c.Next() | |||
| } | |||
| } | |||
| func WssAuth(c *gin.Context) { | |||
| } | |||
| @@ -409,9 +406,5 @@ func SetupContextForToken(c *gin.Context, token *model.Token, parts ...string) e | |||
| } | |||
| common.SetContextKey(c, constant.ContextKeyTokenGroup, token.Group) | |||
| common.SetContextKey(c, constant.ContextKeyTokenCrossGroupRetry, token.CrossGroupRetry) | |||
| // 所有用户都可以通过 sk-{key}:{channelId} 或 sk-{key}-{channelId} 指定渠道 | |||
| if len(parts) > 1 { | |||
| c.Set("specific_channel_id", parts[1]) | |||
| } | |||
| return nil | |||
| } | |||
| @@ -0,0 +1,49 @@ | |||
| package middleware | |||
| import ( | |||
| "testing" | |||
| "github.com/stretchr/testify/assert" | |||
| ) | |||
| // --------------------------------------------------------------------------- | |||
| // parseTokenKey — 渠道选择删除后的验证测试 | |||
| // --------------------------------------------------------------------------- | |||
| func TestParseTokenKey_SimpleKey(t *testing.T) { | |||
| key, parts := parseTokenKey("sk-abc123") | |||
| assert.Equal(t, "abc123", key) | |||
| assert.Equal(t, []string{"abc123"}, parts) | |||
| } | |||
| func TestParseTokenKey_KeyWithDash(t *testing.T) { | |||
| // "sk-abc123-42" 中 "-" 用于分割,parts[0]="abc123" 就是 key | |||
| // parts[1]="42" 不再被解释为 channelId(渠道选择功能已删除) | |||
| key, parts := parseTokenKey("sk-abc123-42") | |||
| assert.Equal(t, "abc123", key) | |||
| assert.Equal(t, []string{"abc123", "42"}, parts) | |||
| // 验证:parts[1] 存在但不会被用于渠道选择 | |||
| // 渠道选择功能已删除,parseTokenKey 仅返回 parts,不解析 channelId | |||
| } | |||
| func TestParseTokenKey_ColonFormat(t *testing.T) { | |||
| // 旧格式 "sk-abc123:42" 中 ":" 不再被解析为 channelId | |||
| // parseTokenKey 只按 "-" 分割,":" 会保留在 parts[0] 中 | |||
| key, parts := parseTokenKey("sk-abc123:42") | |||
| assert.Equal(t, "abc123:42", key) | |||
| assert.Equal(t, []string{"abc123:42"}, parts) | |||
| // 旧行为会返回 key="abc123" 并提取 channelId=42 | |||
| // 新行为:":" 被当作 key 的一部分,不做特殊处理 | |||
| } | |||
| func TestParseTokenKey_EmptyKey(t *testing.T) { | |||
| key, parts := parseTokenKey("sk-") | |||
| assert.Equal(t, "", key) | |||
| assert.Equal(t, []string{""}, parts) | |||
| } | |||
| func TestParseTokenKey_MultipleDashes(t *testing.T) { | |||
| key, parts := parseTokenKey("sk-abc-def-ghi") | |||
| assert.Equal(t, "abc", key) | |||
| assert.Equal(t, []string{"abc", "def", "ghi"}, parts) | |||
| } | |||
| @@ -8,7 +8,10 @@ import ( | |||
| func CORS() gin.HandlerFunc { | |||
| config := cors.DefaultConfig() | |||
| config.AllowAllOrigins = true | |||
| config.AllowAllOrigins = false | |||
| config.AllowOriginFunc = func(origin string) bool { | |||
| return true | |||
| } | |||
| config.AllowCredentials = true | |||
| config.AllowMethods = []string{"GET", "POST", "PUT", "DELETE", "OPTIONS"} | |||
| config.AllowHeaders = []string{"*"} | |||
| @@ -5,7 +5,6 @@ import ( | |||
| "fmt" | |||
| "net/http" | |||
| "slices" | |||
| "strconv" | |||
| "strings" | |||
| "time" | |||
| @@ -23,9 +22,8 @@ import ( | |||
| ) | |||
| type ModelRequest struct { | |||
| Model string `json:"model"` | |||
| Group string `json:"group,omitempty"` | |||
| ChannelId int `json:"channel_id,omitempty"` | |||
| Model string `json:"model"` | |||
| Group string `json:"group,omitempty"` | |||
| } | |||
| func Distribute() func(c *gin.Context) { | |||
| @@ -36,157 +34,97 @@ func Distribute() func(c *gin.Context) { | |||
| abortWithOpenAiMessage(c, http.StatusBadRequest, i18n.T(c, i18n.MsgDistributorInvalidRequest, map[string]any{"Error": err.Error()})) | |||
| return | |||
| } | |||
| channelId, ok := common.GetContextKey(c, constant.ContextKeyTokenSpecificChannelId) | |||
| if ok { | |||
| id, err := strconv.Atoi(channelId.(string)) | |||
| if err != nil { | |||
| abortWithOpenAiMessage(c, http.StatusBadRequest, i18n.T(c, i18n.MsgDistributorInvalidChannelId)) | |||
| // Normal channel selection logic | |||
| // check token model mapping | |||
| modelLimitEnable := common.GetContextKeyBool(c, constant.ContextKeyTokenModelLimitEnabled) | |||
| if modelLimitEnable { | |||
| s, ok := common.GetContextKey(c, constant.ContextKeyTokenModelLimit) | |||
| if !ok { | |||
| // token model limit is empty, all models are not allowed | |||
| abortWithOpenAiMessage(c, http.StatusForbidden, i18n.T(c, i18n.MsgDistributorTokenNoModelAccess)) | |||
| return | |||
| } | |||
| channel, err = model.GetChannelById(id, true) | |||
| if err != nil { | |||
| abortWithOpenAiMessage(c, http.StatusBadRequest, i18n.T(c, i18n.MsgDistributorInvalidChannelId)) | |||
| return | |||
| var tokenModelLimit map[string]bool | |||
| tokenModelLimit, ok = s.(map[string]bool) | |||
| if !ok { | |||
| tokenModelLimit = map[string]bool{} | |||
| } | |||
| if channel.Status != common.ChannelStatusEnabled { | |||
| abortWithOpenAiMessage(c, http.StatusForbidden, i18n.T(c, i18n.MsgDistributorChannelDisabled)) | |||
| matchName := ratio_setting.FormatMatchingModelName(modelRequest.Model) // match gpts & thinking-* | |||
| if _, ok := tokenModelLimit[matchName]; !ok { | |||
| abortWithOpenAiMessage(c, http.StatusForbidden, i18n.T(c, i18n.MsgDistributorTokenModelForbidden, map[string]any{"Model": modelRequest.Model})) | |||
| return | |||
| } | |||
| // 验证指定渠道是否支持请求的模型 | |||
| if shouldSelectChannel && modelRequest.Model != "" { | |||
| usingGroup := common.GetContextKeyString(c, constant.ContextKeyUsingGroup) | |||
| if !model.IsChannelEnabledForGroupModel(usingGroup, modelRequest.Model, channel.Id) { | |||
| abortWithOpenAiMessage(c, http.StatusServiceUnavailable, i18n.T(c, i18n.MsgDistributorNoAvailableChannel, map[string]any{"Group": usingGroup, "Model": modelRequest.Model}), types.ErrorCodeModelNotFound) | |||
| return | |||
| } | |||
| } | |||
| } else { | |||
| // Normal channel selection logic | |||
| // check token model mapping | |||
| modelLimitEnable := common.GetContextKeyBool(c, constant.ContextKeyTokenModelLimitEnabled) | |||
| if modelLimitEnable { | |||
| s, ok := common.GetContextKey(c, constant.ContextKeyTokenModelLimit) | |||
| if !ok { | |||
| // token model limit is empty, all models are not allowed | |||
| abortWithOpenAiMessage(c, http.StatusForbidden, i18n.T(c, i18n.MsgDistributorTokenNoModelAccess)) | |||
| return | |||
| } | |||
| var tokenModelLimit map[string]bool | |||
| tokenModelLimit, ok = s.(map[string]bool) | |||
| if !ok { | |||
| tokenModelLimit = map[string]bool{} | |||
| } | |||
| matchName := ratio_setting.FormatMatchingModelName(modelRequest.Model) // match gpts & thinking-* | |||
| if _, ok := tokenModelLimit[matchName]; !ok { | |||
| abortWithOpenAiMessage(c, http.StatusForbidden, i18n.T(c, i18n.MsgDistributorTokenModelForbidden, map[string]any{"Model": modelRequest.Model})) | |||
| return | |||
| } | |||
| } | |||
| } | |||
| if shouldSelectChannel { | |||
| if modelRequest.Model == "" { | |||
| abortWithOpenAiMessage(c, http.StatusBadRequest, i18n.T(c, i18n.MsgDistributorModelNameRequired)) | |||
| if shouldSelectChannel { | |||
| if modelRequest.Model == "" { | |||
| abortWithOpenAiMessage(c, http.StatusBadRequest, i18n.T(c, i18n.MsgDistributorModelNameRequired)) | |||
| return | |||
| } | |||
| var selectGroup string | |||
| usingGroup := common.GetContextKeyString(c, constant.ContextKeyUsingGroup) | |||
| // check path is /pg/chat/completions | |||
| if strings.HasPrefix(c.Request.URL.Path, "/pg/chat/completions") { | |||
| playgroundRequest := &dto.PlayGroundRequest{} | |||
| err = common.UnmarshalBodyReusable(c, playgroundRequest) | |||
| if err != nil { | |||
| abortWithOpenAiMessage(c, http.StatusBadRequest, i18n.T(c, i18n.MsgDistributorInvalidPlayground, map[string]any{"Error": err.Error()})) | |||
| return | |||
| } | |||
| var selectGroup string | |||
| usingGroup := common.GetContextKeyString(c, constant.ContextKeyUsingGroup) | |||
| // check path is /pg/chat/completions | |||
| if strings.HasPrefix(c.Request.URL.Path, "/pg/chat/completions") { | |||
| playgroundRequest := &dto.PlayGroundRequest{} | |||
| err = common.UnmarshalBodyReusable(c, playgroundRequest) | |||
| if err != nil { | |||
| abortWithOpenAiMessage(c, http.StatusBadRequest, i18n.T(c, i18n.MsgDistributorInvalidPlayground, map[string]any{"Error": err.Error()})) | |||
| if playgroundRequest.Group != "" { | |||
| if !service.GroupInUserUsableGroups(usingGroup, playgroundRequest.Group) && playgroundRequest.Group != usingGroup { | |||
| abortWithOpenAiMessage(c, http.StatusForbidden, i18n.T(c, i18n.MsgDistributorGroupAccessDenied)) | |||
| return | |||
| } | |||
| if playgroundRequest.Group != "" { | |||
| if !service.GroupInUserUsableGroups(usingGroup, playgroundRequest.Group) && playgroundRequest.Group != usingGroup { | |||
| abortWithOpenAiMessage(c, http.StatusForbidden, i18n.T(c, i18n.MsgDistributorGroupAccessDenied)) | |||
| return | |||
| } | |||
| usingGroup = playgroundRequest.Group | |||
| common.SetContextKey(c, constant.ContextKeyUsingGroup, usingGroup) | |||
| } | |||
| usingGroup = playgroundRequest.Group | |||
| common.SetContextKey(c, constant.ContextKeyUsingGroup, usingGroup) | |||
| } | |||
| } | |||
| // 默认通道检查(管理员配置的模型默认通道) | |||
| if channel == nil { | |||
| if defaultChannelId, ok := model.GetDefaultChannelId(modelRequest.Model); ok { | |||
| defaultCh, err := model.CacheGetChannel(defaultChannelId) | |||
| if err != nil || defaultCh == nil { | |||
| common.SysLog(fmt.Sprintf("[Distribute] model=%s default_channel=%d not found in cache, fallback", modelRequest.Model, defaultChannelId)) | |||
| } else if defaultCh.Status != common.ChannelStatusEnabled { | |||
| common.SysLog(fmt.Sprintf("[Distribute] model=%s default_channel=%d disabled(status=%d), fallback", modelRequest.Model, defaultChannelId, defaultCh.Status)) | |||
| } else if usingGroup == "auto" { | |||
| userGroup := common.GetContextKeyString(c, constant.ContextKeyUserGroup) | |||
| autoGroups := service.GetUserAutoGroup(userGroup) | |||
| for _, g := range autoGroups { | |||
| if model.IsChannelEnabledForGroupModel(g, modelRequest.Model, defaultCh.Id) { | |||
| channel = defaultCh | |||
| selectGroup = g | |||
| common.SetContextKey(c, constant.ContextKeyAutoGroup, g) | |||
| common.SysLog(fmt.Sprintf("[Distribute] model=%s using default_channel=%d (auto group=%s)", modelRequest.Model, defaultChannelId, g)) | |||
| break | |||
| } | |||
| } | |||
| if channel == nil { | |||
| common.SysLog(fmt.Sprintf("[Distribute] model=%s default_channel=%d not enabled for any auto group, fallback", modelRequest.Model, defaultChannelId)) | |||
| // 通道亲和性检查 | |||
| if preferredChannelID, found := service.GetPreferredChannelByAffinity(c, modelRequest.Model, usingGroup); found { | |||
| preferred, err := model.CacheGetChannel(preferredChannelID) | |||
| if err == nil && preferred != nil && preferred.Status == common.ChannelStatusEnabled { | |||
| if usingGroup == "auto" { | |||
| userGroup := common.GetContextKeyString(c, constant.ContextKeyUserGroup) | |||
| autoGroups := service.GetUserAutoGroup(userGroup) | |||
| for _, g := range autoGroups { | |||
| if model.IsChannelEnabledForGroupModel(g, modelRequest.Model, preferred.Id) { | |||
| selectGroup = g | |||
| common.SetContextKey(c, constant.ContextKeyAutoGroup, g) | |||
| channel = preferred | |||
| service.MarkChannelAffinityUsed(c, g, preferred.Id) | |||
| break | |||
| } | |||
| } else if model.IsChannelEnabledForGroupModel(usingGroup, modelRequest.Model, defaultCh.Id) { | |||
| channel = defaultCh | |||
| selectGroup = usingGroup | |||
| common.SysLog(fmt.Sprintf("[Distribute] model=%s using default_channel=%d (group=%s)", modelRequest.Model, defaultChannelId, usingGroup)) | |||
| } else { | |||
| common.SysLog(fmt.Sprintf("[Distribute] model=%s default_channel=%d not enabled for group=%s, fallback", modelRequest.Model, defaultChannelId, usingGroup)) | |||
| } | |||
| } else if model.IsChannelEnabledForGroupModel(usingGroup, modelRequest.Model, preferred.Id) { | |||
| channel = preferred | |||
| selectGroup = usingGroup | |||
| service.MarkChannelAffinityUsed(c, usingGroup, preferred.Id) | |||
| } | |||
| } | |||
| } | |||
| // 通道亲和性检查(仅在未选中默认通道时生效) | |||
| if channel == nil { | |||
| if preferredChannelID, found := service.GetPreferredChannelByAffinity(c, modelRequest.Model, usingGroup); found { | |||
| preferred, err := model.CacheGetChannel(preferredChannelID) | |||
| if err == nil && preferred != nil && preferred.Status == common.ChannelStatusEnabled { | |||
| if usingGroup == "auto" { | |||
| userGroup := common.GetContextKeyString(c, constant.ContextKeyUserGroup) | |||
| autoGroups := service.GetUserAutoGroup(userGroup) | |||
| for _, g := range autoGroups { | |||
| if model.IsChannelEnabledForGroupModel(g, modelRequest.Model, preferred.Id) { | |||
| selectGroup = g | |||
| common.SetContextKey(c, constant.ContextKeyAutoGroup, g) | |||
| channel = preferred | |||
| service.MarkChannelAffinityUsed(c, g, preferred.Id) | |||
| break | |||
| } | |||
| } | |||
| } else if model.IsChannelEnabledForGroupModel(usingGroup, modelRequest.Model, preferred.Id) { | |||
| channel = preferred | |||
| selectGroup = usingGroup | |||
| service.MarkChannelAffinityUsed(c, usingGroup, preferred.Id) | |||
| } | |||
| } | |||
| if channel == nil { | |||
| channel, selectGroup, err = service.CacheGetRandomSatisfiedChannel(&service.RetryParam{ | |||
| Ctx: c, | |||
| ModelName: modelRequest.Model, | |||
| TokenGroup: usingGroup, | |||
| Retry: common.GetPointer(0), | |||
| }) | |||
| if err != nil { | |||
| showGroup := usingGroup | |||
| if usingGroup == "auto" { | |||
| showGroup = fmt.Sprintf("auto(%s)", selectGroup) | |||
| } | |||
| message := i18n.T(c, i18n.MsgDistributorGetChannelFailed, map[string]any{"Group": showGroup, "Model": modelRequest.Model, "Error": err.Error()}) | |||
| abortWithOpenAiMessage(c, http.StatusServiceUnavailable, message, types.ErrorCodeModelNotFound) | |||
| return | |||
| } | |||
| if channel == nil { | |||
| channel, selectGroup, err = service.CacheGetRandomSatisfiedChannel(&service.RetryParam{ | |||
| Ctx: c, | |||
| ModelName: modelRequest.Model, | |||
| TokenGroup: usingGroup, | |||
| Retry: common.GetPointer(0), | |||
| }) | |||
| if err != nil { | |||
| showGroup := usingGroup | |||
| if usingGroup == "auto" { | |||
| showGroup = fmt.Sprintf("auto(%s)", selectGroup) | |||
| } | |||
| message := i18n.T(c, i18n.MsgDistributorGetChannelFailed, map[string]any{"Group": showGroup, "Model": modelRequest.Model, "Error": err.Error()}) | |||
| abortWithOpenAiMessage(c, http.StatusServiceUnavailable, message, types.ErrorCodeModelNotFound) | |||
| return | |||
| } | |||
| if channel == nil { | |||
| abortWithOpenAiMessage(c, http.StatusServiceUnavailable, i18n.T(c, i18n.MsgDistributorNoAvailableChannel, map[string]any{"Group": usingGroup, "Model": modelRequest.Model}), types.ErrorCodeModelNotFound) | |||
| return | |||
| } | |||
| abortWithOpenAiMessage(c, http.StatusServiceUnavailable, i18n.T(c, i18n.MsgDistributorNoAvailableChannel, map[string]any{"Group": usingGroup, "Model": modelRequest.Model}), types.ErrorCodeModelNotFound) | |||
| return | |||
| } | |||
| } | |||
| } | |||
| @@ -374,12 +312,6 @@ func getModelRequest(c *gin.Context) (*ModelRequest, bool, error) { | |||
| modelRequest.Group = g | |||
| } | |||
| common.SetContextKey(c, constant.ContextKeyTokenGroup, modelRequest.Group) | |||
| // channel_id: body 优先,fallback 到 header | |||
| if req.ChannelId > 0 { | |||
| common.SetContextKey(c, constant.ContextKeyTokenSpecificChannelId, strconv.Itoa(req.ChannelId)) | |||
| } else if ch := c.GetHeader("X-Channel-Id"); ch != "" { | |||
| common.SetContextKey(c, constant.ContextKeyTokenSpecificChannelId, ch) | |||
| } | |||
| } | |||
| if strings.HasPrefix(c.Request.URL.Path, "/v1/responses/compact") && modelRequest.Model != "" { | |||
| @@ -0,0 +1,142 @@ | |||
| package middleware | |||
| import ( | |||
| "fmt" | |||
| "net/http" | |||
| "slices" | |||
| "strings" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/constant" | |||
| "github.com/QuantumNous/new-api/setting/model_setting" | |||
| "github.com/gin-gonic/gin" | |||
| ) | |||
| // pathToFamily maps URL path prefixes to format families. | |||
| // The middleware uses this to determine the request's protocol family | |||
| // directly from the request path, without relying on context values. | |||
| var pathToFamily = []struct { | |||
| prefix string | |||
| family string | |||
| }{ | |||
| {"/v1/messages", "claude"}, | |||
| {"/v1beta/", "gemini"}, | |||
| {"/v1/models/", "gemini"}, // Gemini relay via /v1/models/*path | |||
| {"/v1/engines/", "gemini"}, // Gemini relay via /v1/engines/:model/embeddings | |||
| // All other /v1/ paths default to "openai" | |||
| } | |||
| // getFamilyFromPath returns the format family for a given request path. | |||
| func getFamilyFromPath(path string) string { | |||
| for _, rule := range pathToFamily { | |||
| if strings.HasPrefix(path, rule.prefix) { | |||
| return rule.family | |||
| } | |||
| } | |||
| if strings.HasPrefix(path, "/v1/") { | |||
| return "openai" | |||
| } | |||
| return "" | |||
| } | |||
| var channelNativeFamilies = map[int][]string{ | |||
| constant.ChannelTypeOpenAI: {"openai"}, | |||
| constant.ChannelTypeAzure: {"openai"}, | |||
| constant.ChannelTypeOllama: {"openai"}, | |||
| constant.ChannelTypeDeepSeek: {"openai"}, | |||
| constant.ChannelTypeOpenRouter: {"openai"}, | |||
| constant.ChannelTypeXinference: {"openai"}, | |||
| constant.ChannelTypeXai: {"openai"}, | |||
| constant.ChannelTypeMistral: {"openai"}, | |||
| constant.ChannelTypeSiliconFlow: {"openai"}, | |||
| constant.ChannelTypePerplexity: {"openai"}, | |||
| constant.ChannelTypeCodex: {"openai"}, | |||
| constant.ChannelTypeMiniMax: {"openai"}, | |||
| constant.ChannelTypeMoonshot: {"openai"}, | |||
| constant.ChannelTypeBaidu: {"openai"}, | |||
| constant.ChannelTypeBaiduV2: {"openai"}, | |||
| constant.ChannelTypeZhipu: {"openai"}, | |||
| constant.ChannelTypeZhipu_v4: {"openai"}, | |||
| constant.ChannelTypeTencent: {"openai"}, | |||
| constant.ChannelTypeXunfei: {"openai"}, | |||
| constant.ChannelTypeCohere: {"openai"}, | |||
| constant.ChannelTypeDify: {"openai"}, | |||
| constant.ChannelTypeCoze: {"openai"}, | |||
| constant.ChannelTypeVolcEngine: {"openai"}, | |||
| constant.ChannelTypeReplicate: {"openai"}, | |||
| constant.ChannelTypeSubmodel: {"openai"}, | |||
| constant.ChannelTypeJina: {"openai"}, | |||
| constant.ChannelTypePaLM: {"openai"}, | |||
| constant.ChannelTypeAnthropic: {"claude"}, | |||
| constant.ChannelTypeAws: {"claude"}, | |||
| constant.ChannelTypeGemini: {"gemini"}, | |||
| constant.ChannelTypeVertexAi: {"openai", "claude", "gemini"}, | |||
| constant.ChannelTypeAli: {"openai", "claude"}, | |||
| } | |||
| var nativeEndpoints = map[string]string{ | |||
| "openai": "/v1/chat/completions", | |||
| "claude": "/v1/messages", | |||
| "gemini": "/v1beta/models/{model}:generateContent", | |||
| } | |||
| func isChannelAllowed(channelType int, family string) bool { | |||
| families, ok := channelNativeFamilies[channelType] | |||
| if !ok { | |||
| return true | |||
| } | |||
| return slices.Contains(families, family) | |||
| } | |||
| func EndpointFormatGuard() gin.HandlerFunc { | |||
| return func(c *gin.Context) { | |||
| if !model_setting.IsEndpointFormatRestrictionEnabled() { | |||
| c.Next() | |||
| return | |||
| } | |||
| family := getFamilyFromPath(c.Request.URL.Path) | |||
| if family == "" { | |||
| c.Next() | |||
| return | |||
| } | |||
| channelType := common.GetContextKeyInt(c, constant.ContextKeyChannelType) | |||
| if channelType == 0 { | |||
| c.Next() | |||
| return | |||
| } | |||
| if isChannelAllowed(channelType, family) { | |||
| c.Next() | |||
| return | |||
| } | |||
| channelTypeName := constant.GetChannelTypeName(channelType) | |||
| var nativeEndpoint string | |||
| if families, ok := channelNativeFamilies[channelType]; ok && len(families) > 0 { | |||
| nativeEndpoint = nativeEndpoints[families[0]] | |||
| } | |||
| c.JSON(http.StatusBadRequest, gin.H{ | |||
| "error": gin.H{ | |||
| "message": fmt.Sprintf( | |||
| "Model is on a %s channel and cannot be accessed via %s endpoints. Please use the native %s endpoint (%s) instead.", | |||
| channelTypeName, family, channelTypeName, nativeEndpoint, | |||
| ), | |||
| "type": "new_api_error", | |||
| "code": "endpoint_format_restricted", | |||
| }, | |||
| }) | |||
| c.Abort() | |||
| } | |||
| } | |||
| // setTestRestrictionEnabled is a test helper. | |||
| func setTestRestrictionEnabled(enabled bool) bool { | |||
| prev := model_setting.IsEndpointFormatRestrictionEnabled() | |||
| model_setting.GetGlobalSettings().EndpointFormatRestrictionEnabled = enabled | |||
| return prev | |||
| } | |||
| @@ -0,0 +1,591 @@ | |||
| package middleware | |||
| import ( | |||
| "encoding/json" | |||
| "net/http/httptest" | |||
| "sync" | |||
| "sync/atomic" | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/constant" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/stretchr/testify/assert" | |||
| "github.com/stretchr/testify/require" | |||
| ) | |||
| // ============================================================================ | |||
| // Group A: getFamilyFromPath 纯函数测试 | |||
| // ============================================================================ | |||
| func TestGetFamilyFromPath_Claude(t *testing.T) { | |||
| assert.Equal(t, "claude", getFamilyFromPath("/v1/messages")) | |||
| } | |||
| func TestGetFamilyFromPath_ClaudeSubpath(t *testing.T) { | |||
| assert.Equal(t, "claude", getFamilyFromPath("/v1/messages/")) | |||
| } | |||
| func TestGetFamilyFromPath_ClaudeBatchPath(t *testing.T) { | |||
| assert.Equal(t, "claude", getFamilyFromPath("/v1/messages/batches")) | |||
| } | |||
| func TestGetFamilyFromPath_Gemini(t *testing.T) { | |||
| assert.Equal(t, "gemini", getFamilyFromPath("/v1beta/models/gemini-pro:generateContent")) | |||
| assert.Equal(t, "gemini", getFamilyFromPath("/v1beta/")) | |||
| } | |||
| func TestGetFamilyFromPath_GeminiCompatPath(t *testing.T) { | |||
| assert.Equal(t, "gemini", getFamilyFromPath("/v1beta/openai/models")) | |||
| } | |||
| func TestGetFamilyFromPath_GeminiStreamPath(t *testing.T) { | |||
| assert.Equal(t, "gemini", getFamilyFromPath("/v1beta/models/gemini-2.0-flash:streamGenerateContent")) | |||
| } | |||
| func TestGetFamilyFromPath_V1ModelsPath(t *testing.T) { | |||
| // /v1/models/*path 用于 Gemini relay,应归类为 gemini 家族 | |||
| assert.Equal(t, "gemini", getFamilyFromPath("/v1/models/gemini-pro:generateContent")) | |||
| assert.Equal(t, "gemini", getFamilyFromPath("/v1/models/gemini-2.0-flash:streamGenerateContent")) | |||
| } | |||
| func TestGetFamilyFromPath_V1EnginesPath(t *testing.T) { | |||
| // /v1/engines/:model/embeddings 用于 Gemini relay,应归类为 gemini 家族 | |||
| assert.Equal(t, "gemini", getFamilyFromPath("/v1/engines/text-embedding-004/embeddings")) | |||
| } | |||
| func TestGetFamilyFromPath_OpenAI(t *testing.T) { | |||
| assert.Equal(t, "openai", getFamilyFromPath("/v1/chat/completions")) | |||
| assert.Equal(t, "openai", getFamilyFromPath("/v1/completions")) | |||
| assert.Equal(t, "openai", getFamilyFromPath("/v1/embeddings")) | |||
| assert.Equal(t, "openai", getFamilyFromPath("/v1/audio/speech")) | |||
| assert.Equal(t, "openai", getFamilyFromPath("/v1/images/generations")) | |||
| assert.Equal(t, "openai", getFamilyFromPath("/v1/rerank")) | |||
| assert.Equal(t, "openai", getFamilyFromPath("/v1/moderations")) | |||
| } | |||
| func TestGetFamilyFromPath_Realtime(t *testing.T) { | |||
| assert.Equal(t, "openai", getFamilyFromPath("/v1/realtime")) | |||
| } | |||
| func TestGetFamilyFromPath_Responses(t *testing.T) { | |||
| assert.Equal(t, "openai", getFamilyFromPath("/v1/responses")) | |||
| assert.Equal(t, "openai", getFamilyFromPath("/v1/responses/compact")) | |||
| } | |||
| func TestGetFamilyFromPath_Unknown(t *testing.T) { | |||
| assert.Equal(t, "", getFamilyFromPath("/api/status")) | |||
| assert.Equal(t, "", getFamilyFromPath("/pg/chat/completions")) | |||
| assert.Equal(t, "", getFamilyFromPath("/")) | |||
| assert.Equal(t, "", getFamilyFromPath("/v1")) | |||
| } | |||
| // ============================================================================ | |||
| // Group B: isChannelAllowed 纯函数测试 | |||
| // ============================================================================ | |||
| func TestIsChannelAllowed_OpenAIChannel_OpenAIFormat(t *testing.T) { | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeOpenAI, "openai")) | |||
| } | |||
| func TestIsChannelAllowed_OpenAIChannel_ClaudeFormat(t *testing.T) { | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeOpenAI, "claude")) | |||
| } | |||
| func TestIsChannelAllowed_OpenAIChannel_GeminiFormat(t *testing.T) { | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeOpenAI, "gemini")) | |||
| } | |||
| func TestIsChannelAllowed_ClaudeChannel_ClaudeFormat(t *testing.T) { | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeAnthropic, "claude")) | |||
| } | |||
| func TestIsChannelAllowed_ClaudeChannel_OpenAIFormat(t *testing.T) { | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeAnthropic, "openai")) | |||
| } | |||
| func TestIsChannelAllowed_GeminiChannel(t *testing.T) { | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeGemini, "gemini")) | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeGemini, "openai")) | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeGemini, "claude")) | |||
| } | |||
| func TestIsChannelAllowed_AzureChannel(t *testing.T) { | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeAzure, "openai")) | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeAzure, "claude")) | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeAzure, "gemini")) | |||
| } | |||
| func TestIsChannelAllowed_DeepSeekChannel(t *testing.T) { | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeDeepSeek, "openai")) | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeDeepSeek, "claude")) | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeDeepSeek, "gemini")) | |||
| } | |||
| func TestIsChannelAllowed_VertexAI_AllFormats(t *testing.T) { | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeVertexAi, "openai")) | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeVertexAi, "claude")) | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeVertexAi, "gemini")) | |||
| } | |||
| func TestIsChannelAllowed_Ali(t *testing.T) { | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeAli, "openai")) | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeAli, "claude")) | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeAli, "gemini")) | |||
| } | |||
| func TestIsChannelAllowed_Midjourney_PassThrough(t *testing.T) { | |||
| // Midjourney 不在 channelNativeFamilies 中,所有格式都应放行 | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeMidjourney, "openai")) | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeMidjourney, "claude")) | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeMidjourney, "gemini")) | |||
| } | |||
| func TestIsChannelAllowed_Kling_PassThrough(t *testing.T) { | |||
| // Kling 不在 channelNativeFamilies 中,所有格式都应放行 | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeKling, "openai")) | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeKling, "claude")) | |||
| } | |||
| func TestIsChannelAllowed_Suno_PassThrough(t *testing.T) { | |||
| // Suno 不在 channelNativeFamilies 中 | |||
| assert.True(t, isChannelAllowed(constant.ChannelTypeSunoAPI, "openai")) | |||
| } | |||
| func TestIsChannelAllowed_UnknownChannel(t *testing.T) { | |||
| assert.True(t, isChannelAllowed(999, "openai")) | |||
| assert.True(t, isChannelAllowed(999, "claude")) | |||
| assert.True(t, isChannelAllowed(999, "gemini")) | |||
| } | |||
| func TestIsChannelAllowed_EmptyFamily(t *testing.T) { | |||
| // 空 family 表示路径不匹配任何已知格式,guard 会直接放行 | |||
| // isChannelAllowed 不会被空 family 调用,但验证行为:openai 在 families 中不含 "" | |||
| assert.False(t, isChannelAllowed(constant.ChannelTypeOpenAI, "")) | |||
| } | |||
| // ============================================================================ | |||
| // Helper: runGuardTest | |||
| // ============================================================================ | |||
| func runGuardTestWithMethod(method, path string, channelType int) (bool, *httptest.ResponseRecorder) { | |||
| called := false | |||
| gin.SetMode(gin.TestMode) | |||
| r := gin.New() | |||
| r.Use(func(c *gin.Context) { | |||
| if channelType >= 0 { | |||
| c.Set(string(constant.ContextKeyChannelType), channelType) | |||
| } | |||
| c.Next() | |||
| }) | |||
| r.Use(EndpointFormatGuard()) | |||
| // 注册所有相关路由 | |||
| r.POST("/v1/chat/completions", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/completions", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/messages", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/embeddings", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/rerank", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/responses", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/responses/compact", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/moderations", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/audio/speech", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/images/generations", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/models/*path", func(ctx *gin.Context) { called = true }) | |||
| r.POST("/v1/engines/:model/embeddings", func(ctx *gin.Context) { called = true }) | |||
| r.GET("/v1/realtime", func(ctx *gin.Context) { called = true }) | |||
| r.Any("/v1beta/models/*path", func(ctx *gin.Context) { called = true }) | |||
| w := httptest.NewRecorder() | |||
| req := httptest.NewRequest(method, path, nil) | |||
| r.ServeHTTP(w, req) | |||
| return called, w | |||
| } | |||
| func runGuardTest(path string, channelType int) (bool, *httptest.ResponseRecorder) { | |||
| return runGuardTestWithMethod("POST", path, channelType) | |||
| } | |||
| // ============================================================================ | |||
| // Group C: 中间件集成测试 | |||
| // ============================================================================ | |||
| func TestEndpointFormatGuard_Disabled(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(false) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/chat/completions", constant.ChannelTypeAnthropic) | |||
| assert.True(t, called, "handler should be called when restriction is disabled") | |||
| } | |||
| func TestEndpointFormatGuard_Disabled_MultipleChannelTypes(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(false) | |||
| defer setTestRestrictionEnabled(orig) | |||
| for _, ch := range []int{constant.ChannelTypeOpenAI, constant.ChannelTypeAnthropic, constant.ChannelTypeGemini, constant.ChannelTypeMidjourney} { | |||
| called, _ := runGuardTest("/v1/chat/completions", ch) | |||
| assert.True(t, called, "all channel types should pass when restriction is disabled") | |||
| } | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_Matched(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/chat/completions", constant.ChannelTypeOpenAI) | |||
| assert.True(t, called, "handler should be called when format matches channel type") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_Mismatched(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, w := runGuardTest("/v1/chat/completions", constant.ChannelTypeAnthropic) | |||
| assert.False(t, called, "handler should NOT be called when format mismatches") | |||
| assert.Equal(t, 400, w.Code) | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_ClaudeEndpoint_ClaudeChannel(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/messages", constant.ChannelTypeAnthropic) | |||
| assert.True(t, called, "claude endpoint + claude channel should be allowed") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_ClaudeEndpoint_OpenAIChannel(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, w := runGuardTest("/v1/messages", constant.ChannelTypeOpenAI) | |||
| assert.False(t, called, "claude endpoint + openai channel should be blocked") | |||
| assert.Equal(t, 400, w.Code) | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_GeminiEndpoint_GeminiChannel(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1beta/models/gemini-pro:generateContent", constant.ChannelTypeGemini) | |||
| assert.True(t, called, "gemini endpoint + gemini channel should be allowed") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_GeminiEndpoint_OpenAIChannel(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, w := runGuardTest("/v1beta/models/gemini-pro:generateContent", constant.ChannelTypeOpenAI) | |||
| assert.False(t, called, "gemini endpoint + openai channel should be blocked") | |||
| assert.Equal(t, 400, w.Code) | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_GeminiChannel_OpenAIEndpoint(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, w := runGuardTest("/v1/chat/completions", constant.ChannelTypeGemini) | |||
| assert.False(t, called, "gemini channel should be blocked on openai endpoint") | |||
| assert.Equal(t, 400, w.Code) | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_GeminiChannel_V1ModelsPath(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/models/gemini-pro:generateContent", constant.ChannelTypeGemini) | |||
| assert.True(t, called, "gemini channel should be allowed on /v1/models/* path (gemini family)") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_DeepSeekAllowedOnOpenAI(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/chat/completions", constant.ChannelTypeDeepSeek) | |||
| assert.True(t, called, "deepseek channel should be allowed on openai endpoint") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_DeepSeekBlockedOnClaude(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, w := runGuardTest("/v1/messages", constant.ChannelTypeDeepSeek) | |||
| assert.False(t, called, "deepseek channel should be blocked on claude endpoint") | |||
| assert.Equal(t, 400, w.Code) | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_MidjourneyPassThrough(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/chat/completions", constant.ChannelTypeMidjourney) | |||
| assert.True(t, called, "midjourney channel should bypass restriction (not LLM)") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_KlingPassThrough(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/chat/completions", constant.ChannelTypeKling) | |||
| assert.True(t, called, "kling channel should bypass restriction (not LLM)") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_RealtimeWebSocket(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTestWithMethod("GET", "/v1/realtime", constant.ChannelTypeOpenAI) | |||
| assert.True(t, called, "openai channel should be allowed on realtime websocket") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_RealtimeWebSocket_Blocked(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, w := runGuardTestWithMethod("GET", "/v1/realtime", constant.ChannelTypeAnthropic) | |||
| assert.False(t, called, "anthropic channel should be blocked on realtime websocket") | |||
| assert.Equal(t, 400, w.Code) | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_ResponsesEndpoint(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/responses", constant.ChannelTypeOpenAI) | |||
| assert.True(t, called, "openai channel should be allowed on /v1/responses") | |||
| called, w := runGuardTest("/v1/responses", constant.ChannelTypeAnthropic) | |||
| assert.False(t, called, "anthropic channel should be blocked on /v1/responses") | |||
| assert.Equal(t, 400, w.Code) | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_AWS_ClaudeAllowed(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/messages", constant.ChannelTypeAws) | |||
| assert.True(t, called, "AWS channel should be allowed for claude endpoint") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_AWS_OpenAIBlocked(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, w := runGuardTest("/v1/chat/completions", constant.ChannelTypeAws) | |||
| assert.False(t, called, "AWS channel should be blocked for openai endpoint") | |||
| assert.Equal(t, 400, w.Code) | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_VertexAI_AllEndpoints(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/chat/completions", constant.ChannelTypeVertexAi) | |||
| assert.True(t, called, "vertexai should be allowed on openai endpoint") | |||
| called, _ = runGuardTest("/v1/messages", constant.ChannelTypeVertexAi) | |||
| assert.True(t, called, "vertexai should be allowed on claude endpoint") | |||
| called, _ = runGuardTest("/v1beta/models/gemini-pro:generateContent", constant.ChannelTypeVertexAi) | |||
| assert.True(t, called, "vertexai should be allowed on gemini endpoint") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_Ali_GeminiBlocked(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/chat/completions", constant.ChannelTypeAli) | |||
| assert.True(t, called, "ali should be allowed on openai endpoint") | |||
| called, _ = runGuardTest("/v1/messages", constant.ChannelTypeAli) | |||
| assert.True(t, called, "ali should be allowed on claude endpoint") | |||
| called, w := runGuardTest("/v1beta/models/gemini-pro:generateContent", constant.ChannelTypeAli) | |||
| assert.False(t, called, "ali should be blocked on gemini endpoint") | |||
| assert.Equal(t, 400, w.Code) | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_NoChannelType(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/chat/completions", -1) | |||
| assert.True(t, called, "handler should be called when no channelType in context") | |||
| } | |||
| func TestEndpointFormatGuard_Enabled_ChannelTypeZero(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| called, _ := runGuardTest("/v1/chat/completions", 0) | |||
| assert.True(t, called, "handler should be called when channelType is 0") | |||
| } | |||
| func TestEndpointFormatGuard_NonV1Path(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| // /api/status 不在 guard 路由组中,无法通过 runGuardTest 测试 | |||
| // 改用直接调用 getFamilyFromPath 验证返回空 | |||
| family := getFamilyFromPath("/api/status") | |||
| assert.Equal(t, "", family, "non-v1 paths should have empty family") | |||
| } | |||
| // ============================================================================ | |||
| // Group D: 错误消息验证 | |||
| // ============================================================================ | |||
| func TestEndpointFormatGuard_ErrorMessage_ClaudeChannel(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| _, w := runGuardTest("/v1/chat/completions", constant.ChannelTypeAnthropic) | |||
| require.Equal(t, 400, w.Code) | |||
| var body map[string]interface{} | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &body)) | |||
| errObj := body["error"].(map[string]interface{}) | |||
| assert.Equal(t, "endpoint_format_restricted", errObj["code"]) | |||
| assert.Equal(t, "new_api_error", errObj["type"]) | |||
| msg := errObj["message"].(string) | |||
| assert.Contains(t, msg, "Anthropic") | |||
| assert.Contains(t, msg, "/v1/messages") | |||
| assert.Contains(t, msg, "openai endpoints") | |||
| } | |||
| func TestEndpointFormatGuard_ErrorMessage_OpenAIChannel(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| _, w := runGuardTest("/v1/messages", constant.ChannelTypeOpenAI) | |||
| require.Equal(t, 400, w.Code) | |||
| var body map[string]interface{} | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &body)) | |||
| errObj := body["error"].(map[string]interface{}) | |||
| assert.Equal(t, "endpoint_format_restricted", errObj["code"]) | |||
| msg := errObj["message"].(string) | |||
| assert.Contains(t, msg, "OpenAI") | |||
| assert.Contains(t, msg, "/v1/chat/completions") | |||
| assert.Contains(t, msg, "claude endpoints") | |||
| } | |||
| func TestEndpointFormatGuard_ErrorMessage_GeminiChannel(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| _, w := runGuardTest("/v1/chat/completions", constant.ChannelTypeGemini) | |||
| require.Equal(t, 400, w.Code) | |||
| var body map[string]interface{} | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &body)) | |||
| errObj := body["error"].(map[string]interface{}) | |||
| assert.Equal(t, "endpoint_format_restricted", errObj["code"]) | |||
| msg := errObj["message"].(string) | |||
| assert.Contains(t, msg, "Gemini") | |||
| assert.Contains(t, msg, "/v1beta/models/{model}:generateContent") | |||
| assert.Contains(t, msg, "openai endpoints") | |||
| } | |||
| func TestEndpointFormatGuard_ErrorMessage_DeepSeekChannel(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| _, w := runGuardTest("/v1/messages", constant.ChannelTypeDeepSeek) | |||
| require.Equal(t, 400, w.Code) | |||
| var body map[string]interface{} | |||
| require.NoError(t, json.Unmarshal(w.Body.Bytes(), &body)) | |||
| errObj := body["error"].(map[string]interface{}) | |||
| assert.Equal(t, "endpoint_format_restricted", errObj["code"]) | |||
| msg := errObj["message"].(string) | |||
| assert.Contains(t, msg, "DeepSeek") | |||
| assert.Contains(t, msg, "/v1/chat/completions") | |||
| } | |||
| // ============================================================================ | |||
| // Group E: 并发测试 | |||
| // ============================================================================ | |||
| func TestEndpointFormatGuard_ConcurrentRequests(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(true) | |||
| defer setTestRestrictionEnabled(orig) | |||
| var allowedCount, blockedCount int64 | |||
| var wg sync.WaitGroup | |||
| for i := 0; i < 20; i++ { | |||
| wg.Add(1) | |||
| go func(idx int) { | |||
| defer wg.Done() | |||
| var ch int | |||
| if idx%2 == 0 { | |||
| ch = constant.ChannelTypeOpenAI | |||
| } else { | |||
| ch = constant.ChannelTypeAnthropic | |||
| } | |||
| called, w := runGuardTest("/v1/chat/completions", ch) | |||
| if idx%2 == 0 { | |||
| if called && w.Code == 200 { | |||
| atomic.AddInt64(&allowedCount, 1) | |||
| } | |||
| } else { | |||
| if !called && w.Code == 400 { | |||
| atomic.AddInt64(&blockedCount, 1) | |||
| } | |||
| } | |||
| }(i) | |||
| } | |||
| wg.Wait() | |||
| assert.Equal(t, int64(10), atomic.LoadInt64(&allowedCount), "10 个 OpenAI 请求应放行") | |||
| assert.Equal(t, int64(10), atomic.LoadInt64(&blockedCount), "10 个 Anthropic 请求应被拦截") | |||
| } | |||
| // ============================================================================ | |||
| // Group F: 功能开关测试 | |||
| // ============================================================================ | |||
| func TestEndpointFormatGuard_RuntimeToggle(t *testing.T) { | |||
| orig := setTestRestrictionEnabled(false) | |||
| defer setTestRestrictionEnabled(orig) | |||
| // 阶段 1: 关闭 → 应放行 | |||
| called, _ := runGuardTest("/v1/chat/completions", constant.ChannelTypeAnthropic) | |||
| assert.True(t, called, "should pass when restriction is OFF") | |||
| // 阶段 2: 开启 → 应拦截 | |||
| setTestRestrictionEnabled(true) | |||
| called, w := runGuardTest("/v1/chat/completions", constant.ChannelTypeAnthropic) | |||
| assert.False(t, called, "should be blocked when restriction is ON") | |||
| assert.Equal(t, 400, w.Code) | |||
| // 阶段 3: 再次关闭 → 应放行 | |||
| setTestRestrictionEnabled(false) | |||
| called, _ = runGuardTest("/v1/chat/completions", constant.ChannelTypeAnthropic) | |||
| assert.True(t, called, "should pass again when restriction is turned OFF") | |||
| } | |||
| // ============================================================================ | |||
| // 辅助测试 | |||
| // ============================================================================ | |||
| func TestEndpointFormatGuard_ContextKeyInt(t *testing.T) { | |||
| gin.SetMode(gin.TestMode) | |||
| w := httptest.NewRecorder() | |||
| c, _ := gin.CreateTestContext(w) | |||
| c.Request = httptest.NewRequest("POST", "/v1/chat/completions", nil) | |||
| c.Set(string(constant.ContextKeyChannelType), 14) | |||
| val := common.GetContextKeyInt(c, constant.ContextKeyChannelType) | |||
| assert.Equal(t, 14, val) | |||
| } | |||
| @@ -66,58 +66,6 @@ func GetAbilitiesByChannelId(channelId int) ([]*Ability, error) { | |||
| return abilities, err | |||
| } | |||
| // GetModelChannelsForGroup 返回指定模型在指定分组下的可用渠道列表及默认渠道ID | |||
| func GetModelChannelsForGroup(modelName string, group string) ([]map[string]any, int, error) { | |||
| var channelIds []int | |||
| err := DB.Model(&Ability{}). | |||
| Where("model = ?", modelName). | |||
| Where("enabled = ?", true). | |||
| Where(commonGroupCol+" = ?", group). | |||
| Distinct("channel_id"). | |||
| Pluck("channel_id", &channelIds).Error | |||
| if err != nil { | |||
| return nil, 0, err | |||
| } | |||
| if len(channelIds) == 0 { | |||
| return []map[string]any{}, 0, nil | |||
| } | |||
| type channelInfo struct { | |||
| Id int `json:"id"` | |||
| Name string `json:"name"` | |||
| PublicName string `json:"public_name"` | |||
| } | |||
| var channels []channelInfo | |||
| err = DB.Table("channels"). | |||
| Where("id IN ? AND status = ?", channelIds, common.ChannelStatusEnabled). | |||
| Select("id, name, public_name"). | |||
| Find(&channels).Error | |||
| if err != nil { | |||
| return nil, 0, err | |||
| } | |||
| defaultChannelId := 0 | |||
| if defaultChId, ok := GetDefaultChannelId(modelName); ok { | |||
| for _, id := range channelIds { | |||
| if id == defaultChId { | |||
| defaultChannelId = defaultChId | |||
| break | |||
| } | |||
| } | |||
| } | |||
| result := make([]map[string]any, 0, len(channels)) | |||
| for _, ch := range channels { | |||
| result = append(result, map[string]any{ | |||
| "id": ch.Id, | |||
| "name": ch.Name, | |||
| "public_name": ch.PublicName, | |||
| }) | |||
| } | |||
| return result, defaultChannelId, nil | |||
| } | |||
| func getPriority(group string, model string, retry int) (int, error) { | |||
| @@ -283,16 +283,6 @@ func GetAllChannels(startIdx int, num int, selectAll bool, idSort bool) ([]*Chan | |||
| return channels, err | |||
| } | |||
| // GetAllChannelsForBinding 获取所有启用的渠道(用于用户绑定渠道) | |||
| // 只返回 id, name, type, remark,不包含敏感信息 | |||
| func GetAllChannelsForBinding() ([]*Channel, error) { | |||
| var channels []*Channel | |||
| err := DB.Select("id, name, public_name, type, remark"). | |||
| Where("status = ?", common.ChannelStatusEnabled). | |||
| Order("priority desc"). | |||
| Find(&channels).Error | |||
| return channels, err | |||
| } | |||
| func GetChannelsByTag(tag string, idSort bool, selectAll bool) ([]*Channel, error) { | |||
| var channels []*Channel | |||
| @@ -1,450 +0,0 @@ | |||
| package model | |||
| import ( | |||
| "fmt" | |||
| "strconv" | |||
| "strings" | |||
| "sync" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/setting/ratio_setting" | |||
| "gorm.io/gorm" | |||
| "gorm.io/gorm/clause" | |||
| ) | |||
| // 渠道定价缓存 | |||
| var ( | |||
| channelPricingCache = make(map[string]*ChannelPricing) // key: "modelName:channelId" | |||
| channelPricingCacheLock sync.RWMutex | |||
| // 默认通道缓存:modelName → channelId | |||
| defaultChannelCache = make(map[string]int) | |||
| defaultChannelCacheLock sync.RWMutex | |||
| ) | |||
| // QuotaType 计费类型 | |||
| const ( | |||
| QuotaTypeByTokens = 0 // 按量计费 | |||
| QuotaTypeByCall = 1 // 按次计费 | |||
| ) | |||
| // ChannelPricing 渠道定价表 | |||
| // 支持同一模型在不同渠道设置不同价格 | |||
| type ChannelPricing struct { | |||
| Id int `json:"id" gorm:"primaryKey"` | |||
| ModelName string `json:"model_name" gorm:"size:128;not null;uniqueIndex:idx_model_channel,priority:1"` | |||
| ChannelId int `json:"channel_id" gorm:"not null;uniqueIndex:idx_model_channel,priority:2;index"` | |||
| QuotaType int `json:"quota_type" gorm:"default:0"` // 0=按量, 1=按次 | |||
| ModelRatio float64 `json:"model_ratio" gorm:"default:0"` | |||
| CompletionRatio float64 `json:"completion_ratio" gorm:"default:0"` | |||
| ModelPrice float64 `json:"model_price" gorm:"default:0"` | |||
| TagIds string `json:"tag_ids" gorm:"type:varchar(255)"` // 逗号分隔的标签ID | |||
| CreatedTime int64 `json:"created_time" gorm:"bigint"` | |||
| UpdatedTime int64 `json:"updated_time" gorm:"bigint"` | |||
| DeletedAt gorm.DeletedAt `json:"-" gorm:"index"` | |||
| // === 新增字段(0 = 未设置,回退全局值) === | |||
| CacheRatio float64 `json:"cache_ratio" gorm:"default:0"` | |||
| CacheCreationRatio float64 `json:"cache_creation_ratio" gorm:"default:0"` | |||
| ImageRatio float64 `json:"image_ratio" gorm:"default:0"` | |||
| AudioRatio float64 `json:"audio_ratio" gorm:"default:0"` | |||
| AudioCompletionRatio float64 `json:"audio_completion_ratio" gorm:"default:0"` | |||
| IsDefault bool `json:"is_default" gorm:"default:false;index"` | |||
| } | |||
| // setCache 写穿透缓存 | |||
| func setCache(key string, cp *ChannelPricing) { | |||
| channelPricingCacheLock.Lock() | |||
| channelPricingCache[key] = cp | |||
| channelPricingCacheLock.Unlock() | |||
| } | |||
| func removeCache(key string) { | |||
| channelPricingCacheLock.Lock() | |||
| delete(channelPricingCache, key) | |||
| channelPricingCacheLock.Unlock() | |||
| } | |||
| // ApplyFields 批量设置定价字段(消除 controller 层的重复赋值) | |||
| func (cp *ChannelPricing) ApplyFields(quotaType int, modelRatio, completionRatio, modelPrice float64, tagIds string, cacheRatio, cacheCreationRatio, imageRatio, audioRatio, audioCompletionRatio float64) { | |||
| cp.QuotaType = quotaType | |||
| cp.ModelRatio = modelRatio | |||
| cp.CompletionRatio = completionRatio | |||
| cp.ModelPrice = modelPrice | |||
| cp.TagIds = tagIds | |||
| cp.CacheRatio = cacheRatio | |||
| cp.CacheCreationRatio = cacheCreationRatio | |||
| cp.ImageRatio = imageRatio | |||
| cp.AudioRatio = audioRatio | |||
| cp.AudioCompletionRatio = audioCompletionRatio | |||
| } | |||
| func (cp *ChannelPricing) Insert() error { | |||
| now := common.GetTimestamp() | |||
| cp.CreatedTime = now | |||
| cp.UpdatedTime = now | |||
| err := DB.Create(cp).Error | |||
| if err == nil { | |||
| setCache(getChannelPricingCacheKey(cp.ModelName, cp.ChannelId), cp) | |||
| if cp.IsDefault { | |||
| setDefaultChannelCache(cp.ModelName, cp.ChannelId) | |||
| } | |||
| } | |||
| return err | |||
| } | |||
| func (cp *ChannelPricing) Update() error { | |||
| cp.UpdatedTime = common.GetTimestamp() | |||
| err := DB.Model(&ChannelPricing{}).Where("id = ?", cp.Id). | |||
| Select("quota_type", "model_ratio", "completion_ratio", "model_price", | |||
| "tag_ids", "cache_ratio", "cache_creation_ratio", "image_ratio", | |||
| "audio_ratio", "audio_completion_ratio", "is_default", "updated_time"). | |||
| Updates(cp).Error | |||
| if err == nil { | |||
| setCache(getChannelPricingCacheKey(cp.ModelName, cp.ChannelId), cp) | |||
| if cp.IsDefault { | |||
| setDefaultChannelCache(cp.ModelName, cp.ChannelId) | |||
| } else { | |||
| clearDefaultChannelCacheIfMatch(cp.ModelName, cp.Id) | |||
| } | |||
| } | |||
| return err | |||
| } | |||
| func (cp *ChannelPricing) Delete() error { | |||
| var existing ChannelPricing | |||
| if err := DB.First(&existing, cp.Id).Error; err != nil { | |||
| return err | |||
| } | |||
| err := DB.Delete(cp).Error | |||
| if err == nil { | |||
| removeCache(getChannelPricingCacheKey(existing.ModelName, existing.ChannelId)) | |||
| if existing.IsDefault { | |||
| clearDefaultChannelCache(existing.ModelName) | |||
| } | |||
| } | |||
| return err | |||
| } | |||
| // GetChannelPricing 获取指定模型在指定渠道的定价 | |||
| func GetChannelPricing(modelName string, channelId int) (*ChannelPricing, error) { | |||
| var cp ChannelPricing | |||
| err := DB.Where("model_name = ? AND channel_id = ?", modelName, channelId).First(&cp).Error | |||
| if err != nil { | |||
| return nil, err | |||
| } | |||
| return &cp, nil | |||
| } | |||
| // GetChannelPricingByModel 获取指定模型的所有渠道定价 | |||
| func GetChannelPricingByModel(modelName string) ([]*ChannelPricing, error) { | |||
| var list []*ChannelPricing | |||
| err := DB.Where("model_name = ?", modelName).Find(&list).Error | |||
| return list, err | |||
| } | |||
| // GetAllChannelPricing 获取所有渠道定价(分页) | |||
| func GetAllChannelPricing(offset int, limit int) ([]*ChannelPricing, int64, error) { | |||
| var list []*ChannelPricing | |||
| var total int64 | |||
| if err := DB.Model(&ChannelPricing{}).Count(&total).Error; err != nil { | |||
| return nil, 0, err | |||
| } | |||
| err := DB.Order("id DESC").Offset(offset).Limit(limit).Find(&list).Error | |||
| return list, total, err | |||
| } | |||
| // BatchUpsertChannelPricing 批量创建或更新渠道定价 | |||
| func BatchUpsertChannelPricing(pricings []*ChannelPricing) error { | |||
| if len(pricings) == 0 { | |||
| return nil | |||
| } | |||
| now := common.GetTimestamp() | |||
| for _, cp := range pricings { | |||
| cp.UpdatedTime = now | |||
| // 仅在 CreatedTime 为空时设置(新记录) | |||
| if cp.CreatedTime == 0 { | |||
| cp.CreatedTime = now | |||
| } | |||
| } | |||
| // 使用 GORM 的 OnConflict 实现 upsert | |||
| // 唯一索引为 idx_model_channel (model_name, channel_id) | |||
| err := DB.Clauses(clause.OnConflict{ | |||
| Columns: []clause.Column{ | |||
| {Name: "model_name"}, | |||
| {Name: "channel_id"}, | |||
| }, | |||
| DoUpdates: clause.AssignmentColumns([]string{ | |||
| "quota_type", | |||
| "model_ratio", | |||
| "completion_ratio", | |||
| "model_price", | |||
| "tag_ids", | |||
| "cache_ratio", | |||
| "cache_creation_ratio", | |||
| "image_ratio", | |||
| "audio_ratio", | |||
| "audio_completion_ratio", | |||
| "updated_time", | |||
| }), | |||
| }).Create(&pricings).Error | |||
| if err == nil { | |||
| for _, cp := range pricings { | |||
| setCache(getChannelPricingCacheKey(cp.ModelName, cp.ChannelId), cp) | |||
| } | |||
| } | |||
| return err | |||
| } | |||
| // getChannelPricingCacheKey 生成缓存键 | |||
| func getChannelPricingCacheKey(modelName string, channelId int) string { | |||
| return fmt.Sprintf("%s:%d", modelName, channelId) | |||
| } | |||
| // GetEffectivePricing 获取有效定价(纯内存查找) | |||
| func GetEffectivePricing(modelName string, channelId int) (*ChannelPricing, bool) { | |||
| key := getChannelPricingCacheKey(modelName, channelId) | |||
| channelPricingCacheLock.RLock() | |||
| cp, ok := channelPricingCache[key] | |||
| channelPricingCacheLock.RUnlock() | |||
| if !ok { | |||
| return nil, false | |||
| } | |||
| return cp, true | |||
| } | |||
| // ParseTagIds 解析逗号分隔的标签ID字符串为 PricingTag 切片 | |||
| func ParseTagIds(tagIds string, tagMap map[int]*PricingTag) []*PricingTag { | |||
| if tagIds == "" { | |||
| return nil | |||
| } | |||
| tags := make([]*PricingTag, 0) | |||
| for _, idStr := range strings.Split(tagIds, ",") { | |||
| if id, err := strconv.Atoi(strings.TrimSpace(idStr)); err == nil { | |||
| if tag, ok := tagMap[id]; ok { | |||
| tags = append(tags, tag) | |||
| } | |||
| } | |||
| } | |||
| return tags | |||
| } | |||
| // LoadChannelPricingCache 全量加载渠道定价到内存(启动时调用) | |||
| func LoadChannelPricingCache() { | |||
| var pricings []*ChannelPricing | |||
| if err := DB.Find(&pricings).Error; err != nil { | |||
| common.SysError("[ChannelPricing] LoadChannelPricingCache failed: " + err.Error()) | |||
| return | |||
| } | |||
| channelPricingCacheLock.Lock() | |||
| channelPricingCache = make(map[string]*ChannelPricing, len(pricings)) | |||
| for _, cp := range pricings { | |||
| key := getChannelPricingCacheKey(cp.ModelName, cp.ChannelId) | |||
| channelPricingCache[key] = cp | |||
| } | |||
| channelPricingCacheLock.Unlock() | |||
| rebuildDefaultChannelCache(pricings) | |||
| common.SysLog(fmt.Sprintf("[ChannelPricing] cache loaded %d records", len(pricings))) | |||
| } | |||
| // ChannelPricingWithChannel 带渠道信息的定价响应 | |||
| type ChannelPricingWithChannel struct { | |||
| Id int `json:"id"` | |||
| ChannelId int `json:"channel_id"` | |||
| ChannelName string `json:"channel_name"` | |||
| ChannelPublicName string `json:"channel_public_name"` | |||
| ChannelType int `json:"channel_type"` | |||
| TagIds string `json:"tag_ids" gorm:"column:tag_ids"` // 渠道定价的标签ID列表(逗号分隔) | |||
| Tags []*PricingTag `json:"tags" gorm:"-"` // 渠道定价的标签详情(不参与数据库扫描) | |||
| QuotaType int `json:"quota_type"` // 0=按量, 1=按次 | |||
| ModelRatio float64 `json:"model_ratio"` | |||
| CompletionRatio float64 `json:"completion_ratio"` | |||
| ModelPrice float64 `json:"model_price"` | |||
| HasCustomPricing bool `json:"has_custom_pricing"` // 是否有自定义定价 | |||
| CacheRatio float64 `json:"cache_ratio"` | |||
| CacheCreationRatio float64 `json:"cache_creation_ratio"` | |||
| ImageRatio float64 `json:"image_ratio"` | |||
| AudioRatio float64 `json:"audio_ratio"` | |||
| AudioCompletionRatio float64 `json:"audio_completion_ratio"` | |||
| IsDefault bool `json:"is_default"` | |||
| UserRatio float64 `json:"user_ratio"` | |||
| } | |||
| // GetChannelPricingByModelWithChannelInfo 获取指定模型的渠道定价(带渠道信息) | |||
| // 返回所有支持该模型的渠道,对于没有渠道定价的渠道使用全局默认价格 | |||
| func GetChannelPricingByModelWithChannelInfo(modelName string) ([]*ChannelPricingWithChannel, error) { | |||
| var results []*ChannelPricingWithChannel | |||
| // 获取全局默认价格 | |||
| globalModelRatio, hasRatio, _ := ratio_setting.GetModelRatio(modelName) | |||
| globalModelPrice, hasPrice := ratio_setting.GetModelPrice(modelName, false) | |||
| globalCompletionRatio := ratio_setting.GetCompletionRatio(modelName) | |||
| // 确定默认计费类型 | |||
| var defaultQuotaType int | |||
| if hasPrice { | |||
| defaultQuotaType = QuotaTypeByCall | |||
| } else { | |||
| defaultQuotaType = QuotaTypeByTokens | |||
| } | |||
| // 如果没有全局价格,设置默认值 | |||
| if !hasRatio { | |||
| globalModelRatio = 0 | |||
| } | |||
| if !hasPrice { | |||
| globalModelPrice = 0 | |||
| } | |||
| // 查询所有支持该模型的渠道,左连接渠道定价表 | |||
| // 高级字段(cache/image/audio)不回退全局值,直接返回 0 | |||
| err := DB.Table("abilities"). | |||
| Select(`abilities.channel_id, channels.name as channel_name, channels.public_name as channel_public_name, channels.type as channel_type, | |||
| COALESCE(channel_pricings.quota_type, ?) as quota_type, | |||
| COALESCE(channel_pricings.model_ratio, ?) as model_ratio, | |||
| COALESCE(channel_pricings.completion_ratio, ?) as completion_ratio, | |||
| COALESCE(channel_pricings.model_price, ?) as model_price, | |||
| channel_pricings.id as id, | |||
| channel_pricings.tag_ids as tag_ids, | |||
| COALESCE(channel_pricings.cache_ratio, 0) as cache_ratio, | |||
| COALESCE(channel_pricings.cache_creation_ratio, 0) as cache_creation_ratio, | |||
| COALESCE(channel_pricings.image_ratio, 0) as image_ratio, | |||
| COALESCE(channel_pricings.audio_ratio, 0) as audio_ratio, | |||
| COALESCE(channel_pricings.audio_completion_ratio, 0) as audio_completion_ratio, | |||
| COALESCE(channel_pricings.is_default, false) as is_default, | |||
| (channel_pricings.id IS NOT NULL) as has_custom_pricing`, | |||
| defaultQuotaType, globalModelRatio, globalCompletionRatio, globalModelPrice). | |||
| Joins("LEFT JOIN channels ON abilities.channel_id = channels.id"). | |||
| Joins("LEFT JOIN channel_pricings ON abilities.channel_id = channel_pricings.channel_id AND channel_pricings.model_name = ? AND channel_pricings.deleted_at IS NULL", modelName). | |||
| Where("abilities.model = ?", modelName). | |||
| Where("abilities.enabled = ?", true). | |||
| Where("channels.status = ?", 1). // 只显示启用的渠道 | |||
| Group("abilities.channel_id, channels.name, channels.public_name, channels.type, channel_pricings.quota_type, channel_pricings.model_ratio, channel_pricings.completion_ratio, channel_pricings.model_price, channel_pricings.id, channel_pricings.tag_ids, channel_pricings.cache_ratio, channel_pricings.cache_creation_ratio, channel_pricings.image_ratio, channel_pricings.audio_ratio, channel_pricings.audio_completion_ratio, channel_pricings.is_default"). | |||
| Scan(&results).Error | |||
| if err != nil { | |||
| return nil, err | |||
| } | |||
| // 获取所有定价标签 | |||
| allTags, err := GetAllPricingTags() | |||
| if err != nil { | |||
| return results, nil // 如果获取标签失败,仍然返回基础结果 | |||
| } | |||
| // 建立标签ID到标签的映射 | |||
| tagMap := make(map[int]*PricingTag) | |||
| for _, tag := range allTags { | |||
| tagMap[tag.Id] = tag | |||
| } | |||
| // 为每个渠道定价填充标签 | |||
| for _, result := range results { | |||
| result.Tags = ParseTagIds(result.TagIds, tagMap) | |||
| } | |||
| return results, nil | |||
| } | |||
| // === 默认通道缓存 === | |||
| // syncIsDefaultToCache 将默认标记的变更同步到 channelPricingCache | |||
| func syncIsDefaultToCache(modelName string, channelId int, isDefault bool) { | |||
| channelPricingCacheLock.Lock() | |||
| for key, cp := range channelPricingCache { | |||
| if cp.ModelName == modelName { | |||
| if isDefault { | |||
| cp.IsDefault = cp.ChannelId == channelId | |||
| } else { | |||
| cp.IsDefault = false | |||
| } | |||
| } | |||
| channelPricingCache[key] = cp | |||
| } | |||
| channelPricingCacheLock.Unlock() | |||
| } | |||
| // setDefaultChannelCache 设置默认通道缓存 | |||
| func setDefaultChannelCache(modelName string, channelId int) { | |||
| defaultChannelCacheLock.Lock() | |||
| defaultChannelCache[modelName] = channelId | |||
| defaultChannelCacheLock.Unlock() | |||
| } | |||
| // clearDefaultChannelCache 清除指定模型的默认通道缓存 | |||
| func clearDefaultChannelCache(modelName string) { | |||
| defaultChannelCacheLock.Lock() | |||
| delete(defaultChannelCache, modelName) | |||
| defaultChannelCacheLock.Unlock() | |||
| } | |||
| // clearDefaultChannelCacheIfMatch 如果默认通道的定价记录 ID 匹配则清除 | |||
| func clearDefaultChannelCacheIfMatch(modelName string, pricingId int) { | |||
| defaultChannelCacheLock.RLock() | |||
| cachedId, ok := defaultChannelCache[modelName] | |||
| defaultChannelCacheLock.RUnlock() | |||
| if !ok { | |||
| return | |||
| } | |||
| // 需要通过缓存找到对应的 pricing 来比对 | |||
| key := getChannelPricingCacheKey(modelName, cachedId) | |||
| channelPricingCacheLock.RLock() | |||
| cp, exists := channelPricingCache[key] | |||
| channelPricingCacheLock.RUnlock() | |||
| if exists && cp.Id == pricingId { | |||
| clearDefaultChannelCache(modelName) | |||
| } | |||
| } | |||
| // rebuildDefaultChannelCache 从全量数据构建默认通道缓存(启动时调用) | |||
| func rebuildDefaultChannelCache(pricings []*ChannelPricing) { | |||
| defaultChannelCacheLock.Lock() | |||
| defaultChannelCache = make(map[string]int) | |||
| for _, cp := range pricings { | |||
| if cp.IsDefault { | |||
| defaultChannelCache[cp.ModelName] = cp.ChannelId | |||
| } | |||
| } | |||
| defaultChannelCacheLock.Unlock() | |||
| common.SysLog(fmt.Sprintf("[ChannelPricing] default channel cache loaded %d records", len(defaultChannelCache))) | |||
| } | |||
| // GetDefaultChannelId 获取指定模型的默认通道 ID(纯内存读) | |||
| func GetDefaultChannelId(modelName string) (int, bool) { | |||
| defaultChannelCacheLock.RLock() | |||
| id, ok := defaultChannelCache[modelName] | |||
| defaultChannelCacheLock.RUnlock() | |||
| return id, ok | |||
| } | |||
| // SetDefaultChannel 设置指定模型的默认通道(事务保证互斥) | |||
| func SetDefaultChannel(modelName string, channelId int) error { | |||
| return DB.Transaction(func(tx *gorm.DB) error { | |||
| // 清除该模型所有现有的默认标记 | |||
| if err := tx.Model(&ChannelPricing{}). | |||
| Where("model_name = ? AND is_default = ?", modelName, true). | |||
| Update("is_default", false).Error; err != nil { | |||
| return err | |||
| } | |||
| // 设置新的默认 | |||
| if err := tx.Model(&ChannelPricing{}). | |||
| Where("model_name = ? AND channel_id = ?", modelName, channelId). | |||
| Update("is_default", true).Error; err != nil { | |||
| return err | |||
| } | |||
| // 更新缓存 | |||
| setDefaultChannelCache(modelName, channelId) | |||
| syncIsDefaultToCache(modelName, channelId, true) | |||
| return nil | |||
| }) | |||
| } | |||
| // ClearDefaultChannel 清除指定模型的默认通道标记 | |||
| func ClearDefaultChannel(modelName string) error { | |||
| err := DB.Model(&ChannelPricing{}). | |||
| Where("model_name = ? AND is_default = ?", modelName, true). | |||
| Update("is_default", false).Error | |||
| if err == nil { | |||
| clearDefaultChannelCache(modelName) | |||
| syncIsDefaultToCache(modelName, 0, false) | |||
| } | |||
| return err | |||
| } | |||
| @@ -1,242 +0,0 @@ | |||
| package model | |||
| import ( | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/assert" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| func setupChannelPricingDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, _ := db.DB() | |||
| sqlDB.SetMaxOpenConns(1) | |||
| origDB := DB | |||
| DB = db | |||
| common.UsingSQLite = true | |||
| common.RedisEnabled = false | |||
| require.NoError(t, db.AutoMigrate(&ChannelPricing{})) | |||
| t.Cleanup(func() { | |||
| DB = origDB | |||
| sqlDB.Close() | |||
| }) | |||
| return db | |||
| } | |||
| func TestCacheWriteThrough_Insert(t *testing.T) { | |||
| setupChannelPricingDB(t) | |||
| cp := &ChannelPricing{ | |||
| ModelName: "test-insert-model", | |||
| ChannelId: 9001, | |||
| QuotaType: QuotaTypeByTokens, | |||
| ModelRatio: 1.0, | |||
| CacheRatio: 0.8, | |||
| ImageRatio: 1.2, | |||
| } | |||
| require.NoError(t, cp.Insert()) | |||
| defer cp.Delete() | |||
| found, ok := GetEffectivePricing("test-insert-model", 9001) | |||
| assert.True(t, ok) | |||
| assert.Equal(t, 0.8, found.CacheRatio) | |||
| assert.Equal(t, 1.2, found.ImageRatio) | |||
| } | |||
| func TestCacheWriteThrough_Update(t *testing.T) { | |||
| setupChannelPricingDB(t) | |||
| cp := &ChannelPricing{ | |||
| ModelName: "test-update-model", | |||
| ChannelId: 9002, | |||
| QuotaType: QuotaTypeByTokens, | |||
| ModelRatio: 1.0, | |||
| } | |||
| require.NoError(t, cp.Insert()) | |||
| defer cp.Delete() | |||
| cp.CacheRatio = 0.9 | |||
| cp.AudioRatio = 1.5 | |||
| require.NoError(t, cp.Update()) | |||
| found, ok := GetEffectivePricing("test-update-model", 9002) | |||
| assert.True(t, ok) | |||
| assert.Equal(t, 0.9, found.CacheRatio) | |||
| assert.Equal(t, 1.5, found.AudioRatio) | |||
| } | |||
| func TestCacheWriteThrough_Delete(t *testing.T) { | |||
| setupChannelPricingDB(t) | |||
| cp := &ChannelPricing{ | |||
| ModelName: "test-delete-model", | |||
| ChannelId: 9003, | |||
| QuotaType: QuotaTypeByTokens, | |||
| ModelRatio: 1.0, | |||
| } | |||
| require.NoError(t, cp.Insert()) | |||
| require.NoError(t, cp.Delete()) | |||
| found, ok := GetEffectivePricing("test-delete-model", 9003) | |||
| assert.False(t, ok) | |||
| assert.Nil(t, found) | |||
| } | |||
| func TestExtendedFields_DefaultZero(t *testing.T) { | |||
| setupChannelPricingDB(t) | |||
| cp := &ChannelPricing{ | |||
| ModelName: "test-default-model", | |||
| ChannelId: 9004, | |||
| QuotaType: QuotaTypeByTokens, | |||
| ModelRatio: 1.0, | |||
| } | |||
| require.NoError(t, cp.Insert()) | |||
| defer cp.Delete() | |||
| found, ok := GetEffectivePricing("test-default-model", 9004) | |||
| assert.True(t, ok) | |||
| assert.Equal(t, 0.0, found.CacheRatio) | |||
| assert.Equal(t, 0.0, found.CacheCreationRatio) | |||
| assert.Equal(t, 0.0, found.ImageRatio) | |||
| assert.Equal(t, 0.0, found.AudioRatio) | |||
| assert.Equal(t, 0.0, found.AudioCompletionRatio) | |||
| } | |||
| func TestGetEffectivePricing_NotFound(t *testing.T) { | |||
| setupChannelPricingDB(t) | |||
| found, ok := GetEffectivePricing("nonexistent-model-xyz", 99999) | |||
| assert.False(t, ok) | |||
| assert.Nil(t, found) | |||
| } | |||
| // === 默认通道测试 === | |||
| func TestDefaultChannel_SetAndGet(t *testing.T) { | |||
| setupChannelPricingDB(t) | |||
| // 创建两条定价记录 | |||
| cp1 := &ChannelPricing{ModelName: "default-test-model", ChannelId: 100, QuotaType: QuotaTypeByTokens, ModelRatio: 1.0} | |||
| cp2 := &ChannelPricing{ModelName: "default-test-model", ChannelId: 200, QuotaType: QuotaTypeByTokens, ModelRatio: 2.0} | |||
| require.NoError(t, cp1.Insert()) | |||
| require.NoError(t, cp2.Insert()) | |||
| t.Cleanup(func() { cp1.Delete(); cp2.Delete() }) | |||
| // 初始没有默认 | |||
| _, ok := GetDefaultChannelId("default-test-model") | |||
| assert.False(t, ok) | |||
| // 设置通道 100 为默认 | |||
| require.NoError(t, SetDefaultChannel("default-test-model", 100)) | |||
| id, ok := GetDefaultChannelId("default-test-model") | |||
| assert.True(t, ok) | |||
| assert.Equal(t, 100, id) | |||
| // 切换默认到通道 200 | |||
| require.NoError(t, SetDefaultChannel("default-test-model", 200)) | |||
| id, ok = GetDefaultChannelId("default-test-model") | |||
| assert.True(t, ok) | |||
| assert.Equal(t, 200, id) | |||
| // 验证旧的默认标记被清除 | |||
| found, _ := GetEffectivePricing("default-test-model", 100) | |||
| assert.False(t, found.IsDefault) | |||
| found, _ = GetEffectivePricing("default-test-model", 200) | |||
| assert.True(t, found.IsDefault) | |||
| } | |||
| func TestDefaultChannel_Clear(t *testing.T) { | |||
| setupChannelPricingDB(t) | |||
| cp := &ChannelPricing{ModelName: "clear-test-model", ChannelId: 300, QuotaType: QuotaTypeByTokens, ModelRatio: 1.0} | |||
| require.NoError(t, cp.Insert()) | |||
| t.Cleanup(func() { cp.Delete() }) | |||
| require.NoError(t, SetDefaultChannel("clear-test-model", 300)) | |||
| _, ok := GetDefaultChannelId("clear-test-model") | |||
| assert.True(t, ok) | |||
| require.NoError(t, ClearDefaultChannel("clear-test-model")) | |||
| _, ok = GetDefaultChannelId("clear-test-model") | |||
| assert.False(t, ok) | |||
| } | |||
| func TestDefaultChannel_DeleteClearsCache(t *testing.T) { | |||
| setupChannelPricingDB(t) | |||
| cp := &ChannelPricing{ModelName: "delete-test-model", ChannelId: 400, QuotaType: QuotaTypeByTokens, ModelRatio: 1.0, IsDefault: true} | |||
| require.NoError(t, cp.Insert()) | |||
| id, ok := GetDefaultChannelId("delete-test-model") | |||
| assert.True(t, ok) | |||
| assert.Equal(t, 400, id) | |||
| // 删除记录后缓存应清除 | |||
| require.NoError(t, cp.Delete()) | |||
| _, ok = GetDefaultChannelId("delete-test-model") | |||
| assert.False(t, ok) | |||
| } | |||
| func TestDefaultChannel_LoadCache(t *testing.T) { | |||
| db := setupChannelPricingDB(t) | |||
| // 直接插入数据(绕过缓存) | |||
| db.Create(&ChannelPricing{ModelName: "load-model", ChannelId: 500, ModelRatio: 1.0, IsDefault: true}) | |||
| db.Create(&ChannelPricing{ModelName: "load-model", ChannelId: 501, ModelRatio: 2.0, IsDefault: false}) | |||
| db.Create(&ChannelPricing{ModelName: "other-model", ChannelId: 502, ModelRatio: 1.0, IsDefault: true}) | |||
| // 全量加载缓存 | |||
| LoadChannelPricingCache() | |||
| id, ok := GetDefaultChannelId("load-model") | |||
| assert.True(t, ok) | |||
| assert.Equal(t, 500, id) | |||
| id, ok = GetDefaultChannelId("other-model") | |||
| assert.True(t, ok) | |||
| assert.Equal(t, 502, id) | |||
| // 无默认的模型 | |||
| _, ok = GetDefaultChannelId("nonexistent") | |||
| assert.False(t, ok) | |||
| } | |||
| func TestDefaultChannel_InsertWithDefault(t *testing.T) { | |||
| setupChannelPricingDB(t) | |||
| cp := &ChannelPricing{ModelName: "insert-default-model", ChannelId: 600, ModelRatio: 1.0, IsDefault: true} | |||
| require.NoError(t, cp.Insert()) | |||
| t.Cleanup(func() { cp.Delete() }) | |||
| id, ok := GetDefaultChannelId("insert-default-model") | |||
| assert.True(t, ok) | |||
| assert.Equal(t, 600, id) | |||
| } | |||
| func TestDefaultChannel_UpdateWithDefault(t *testing.T) { | |||
| setupChannelPricingDB(t) | |||
| cp1 := &ChannelPricing{ModelName: "update-default-model", ChannelId: 700, ModelRatio: 1.0, IsDefault: true} | |||
| cp2 := &ChannelPricing{ModelName: "update-default-model", ChannelId: 701, ModelRatio: 2.0} | |||
| require.NoError(t, cp1.Insert()) | |||
| require.NoError(t, cp2.Insert()) | |||
| t.Cleanup(func() { cp1.Delete(); cp2.Delete() }) | |||
| // cp1 是默认,通过 Update 把 cp2 设为默认 | |||
| cp2.IsDefault = true | |||
| require.NoError(t, cp2.Update()) | |||
| id, ok := GetDefaultChannelId("update-default-model") | |||
| assert.True(t, ok) | |||
| assert.Equal(t, 701, id) | |||
| } | |||
| @@ -0,0 +1,187 @@ | |||
| package model | |||
| import ( | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/assert" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| func setupChannelSelectDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, err := db.DB() | |||
| require.NoError(t, err) | |||
| sqlDB.SetMaxOpenConns(1) | |||
| origDB := DB | |||
| origUsingSQLite := common.UsingSQLite | |||
| origRedisEnabled := common.RedisEnabled | |||
| origMemoryCache := common.MemoryCacheEnabled | |||
| origGroupCol := commonGroupCol | |||
| origTrueVal := commonTrueVal | |||
| DB = db | |||
| common.UsingSQLite = true | |||
| common.RedisEnabled = false | |||
| common.MemoryCacheEnabled = false | |||
| // Set column quoting for SQLite | |||
| commonGroupCol = "`group`" | |||
| commonTrueVal = "1" | |||
| require.NoError(t, db.AutoMigrate(&Channel{}, &Ability{})) | |||
| t.Cleanup(func() { | |||
| DB = origDB | |||
| common.UsingSQLite = origUsingSQLite | |||
| common.RedisEnabled = origRedisEnabled | |||
| common.MemoryCacheEnabled = origMemoryCache | |||
| commonGroupCol = origGroupCol | |||
| commonTrueVal = origTrueVal | |||
| sqlDB.Close() | |||
| }) | |||
| return db | |||
| } | |||
| func int64Ptr(v int64) *int64 { return &v } | |||
| func intPtr(v int) *int { return &v } | |||
| func uintPtr(v uint) *uint { return &v } | |||
| func createChannelForTest(t *testing.T, db *gorm.DB, id int, name, group, models string, priority int64) { | |||
| t.Helper() | |||
| ch := Channel{ | |||
| Id: id, | |||
| Type: 1, | |||
| Key: "test-key-" + name, | |||
| Status: common.ChannelStatusEnabled, | |||
| Name: name, | |||
| Group: group, | |||
| Models: models, | |||
| Priority: int64Ptr(priority), | |||
| Weight: uintPtr(10), | |||
| AutoBan: intPtr(1), | |||
| CreatedTime: 1, | |||
| } | |||
| require.NoError(t, db.Create(&ch).Error) | |||
| } | |||
| func createAbilityForTest(t *testing.T, db *gorm.DB, group, modelName string, channelId int, priority int64, enabled bool) { | |||
| t.Helper() | |||
| ability := Ability{ | |||
| Group: group, | |||
| Model: modelName, | |||
| ChannelId: channelId, | |||
| Enabled: enabled, | |||
| Priority: int64Ptr(priority), | |||
| Weight: 10, | |||
| } | |||
| require.NoError(t, db.Create(&ability).Error) | |||
| } | |||
| // --- GetChannel (DB path) tests --- | |||
| func TestGetChannel_BasicSelection(t *testing.T) { | |||
| db := setupChannelSelectDB(t) | |||
| createChannelForTest(t, db, 1, "ch1", "default", "gpt-4", 10) | |||
| createAbilityForTest(t, db, "default", "gpt-4", 1, 10, true) | |||
| ch, err := GetChannel("default", "gpt-4", 0) | |||
| require.NoError(t, err) | |||
| assert.NotNil(t, ch) | |||
| assert.Equal(t, 1, ch.Id) | |||
| } | |||
| func TestGetChannel_NoChannelForModel(t *testing.T) { | |||
| db := setupChannelSelectDB(t) | |||
| createChannelForTest(t, db, 1, "ch1", "default", "gpt-3.5", 10) | |||
| createAbilityForTest(t, db, "default", "gpt-3.5", 1, 10, true) | |||
| ch, err := GetChannel("default", "gpt-4", 0) | |||
| assert.NoError(t, err) | |||
| assert.Nil(t, ch) | |||
| } | |||
| func TestGetChannel_PrioritySelection(t *testing.T) { | |||
| db := setupChannelSelectDB(t) | |||
| // Two channels with different priorities | |||
| createChannelForTest(t, db, 1, "ch-low", "default", "gpt-4", 0) | |||
| createChannelForTest(t, db, 2, "ch-high", "default", "gpt-4", 10) | |||
| createAbilityForTest(t, db, "default", "gpt-4", 1, 0, true) | |||
| createAbilityForTest(t, db, "default", "gpt-4", 2, 10, true) | |||
| // Retry=0 should get highest priority (10) | |||
| ch, err := GetChannel("default", "gpt-4", 0) | |||
| require.NoError(t, err) | |||
| assert.NotNil(t, ch) | |||
| assert.Equal(t, 2, ch.Id) // ch-high has priority 10 | |||
| // Retry=1 should get next priority (0) | |||
| ch, err = GetChannel("default", "gpt-4", 1) | |||
| require.NoError(t, err) | |||
| assert.NotNil(t, ch) | |||
| assert.Equal(t, 1, ch.Id) // ch-low has priority 0 | |||
| } | |||
| func TestGetChannel_DifferentGroups(t *testing.T) { | |||
| db := setupChannelSelectDB(t) | |||
| // Channel in "default" group | |||
| createChannelForTest(t, db, 1, "ch-default", "default", "gpt-4", 10) | |||
| createAbilityForTest(t, db, "default", "gpt-4", 1, 10, true) | |||
| // Channel in "vip" group | |||
| createChannelForTest(t, db, 2, "ch-vip", "vip", "gpt-4", 10) | |||
| createAbilityForTest(t, db, "vip", "gpt-4", 2, 10, true) | |||
| // Query default group | |||
| ch, err := GetChannel("default", "gpt-4", 0) | |||
| require.NoError(t, err) | |||
| assert.Equal(t, 1, ch.Id) | |||
| // Query vip group | |||
| ch, err = GetChannel("vip", "gpt-4", 0) | |||
| require.NoError(t, err) | |||
| assert.Equal(t, 2, ch.Id) | |||
| } | |||
| func TestGetChannel_DisabledAbility(t *testing.T) { | |||
| db := setupChannelSelectDB(t) | |||
| createChannelForTest(t, db, 1, "ch1", "default", "gpt-4", 10) | |||
| createAbilityForTest(t, db, "default", "gpt-4", 1, 10, false) // disabled | |||
| ch, err := GetChannel("default", "gpt-4", 0) | |||
| assert.NoError(t, err) | |||
| assert.Nil(t, ch) // no enabled ability | |||
| } | |||
| func TestGetChannel_MultipleChannelsWeighted(t *testing.T) { | |||
| db := setupChannelSelectDB(t) | |||
| createChannelForTest(t, db, 1, "ch1", "default", "gpt-4", 10) | |||
| createChannelForTest(t, db, 2, "ch2", "default", "gpt-4", 10) | |||
| createAbilityForTest(t, db, "default", "gpt-4", 1, 10, true) | |||
| createAbilityForTest(t, db, "default", "gpt-4", 2, 10, true) | |||
| // Run multiple times to verify both channels can be selected | |||
| found := make(map[int]bool) | |||
| for i := 0; i < 50; i++ { | |||
| ch, err := GetChannel("default", "gpt-4", 0) | |||
| require.NoError(t, err) | |||
| require.NotNil(t, ch) | |||
| found[ch.Id] = true | |||
| } | |||
| // With equal weights, both should appear at least once in 50 tries | |||
| assert.True(t, found[1], "channel 1 should be selected at least once") | |||
| assert.True(t, found[2], "channel 2 should be selected at least once") | |||
| } | |||
| @@ -104,6 +104,15 @@ func GetCustomOAuthProviderBySlug(slug string) (*CustomOAuthProvider, error) { | |||
| return &provider, nil | |||
| } | |||
| func GetCustomOAuthProvidersBySlugs(slugs []string) ([]*CustomOAuthProvider, error) { | |||
| var providers []*CustomOAuthProvider | |||
| if len(slugs) == 0 { | |||
| return providers, nil | |||
| } | |||
| err := DB.Where("slug IN ?", slugs).Order("id asc").Find(&providers).Error | |||
| return providers, err | |||
| } | |||
| // CreateCustomOAuthProvider creates a new custom OAuth provider | |||
| func CreateCustomOAuthProvider(provider *CustomOAuthProvider) error { | |||
| if err := validateCustomOAuthProvider(provider); err != nil { | |||
| @@ -69,7 +69,7 @@ func formatUserLogs(logs []*Log, startIdx int) { | |||
| } | |||
| func GetLogByTokenId(tokenId int) (logs []*Log, err error) { | |||
| err = LOG_DB.Model(&Log{}).Where("token_id = ?", tokenId).Order("id desc").Limit(common.MaxRecentItems).Find(&logs).Error | |||
| err = LOG_DB.Model(&Log{}).Where("token_id = ?", tokenId).Order("created_at desc, id desc").Limit(common.MaxRecentItems).Find(&logs).Error | |||
| formatUserLogs(logs, 0) | |||
| return logs, err | |||
| } | |||
| @@ -291,11 +291,11 @@ func GetAllLogs(logType int, startTimestamp int64, endTimestamp int64, modelName | |||
| if group != "" { | |||
| tx = tx.Where("logs."+logGroupCol+" = ?", group) | |||
| } | |||
| err = tx.Model(&Log{}).Count(&total).Error | |||
| err = tx.Model(&Log{}).Limit(logSearchCountLimit).Count(&total).Error | |||
| if err != nil { | |||
| return nil, 0, err | |||
| } | |||
| err = tx.Order("logs.id desc").Limit(num).Offset(startIdx).Find(&logs).Error | |||
| err = tx.Order("logs.created_at desc, logs.id desc").Limit(num).Offset(startIdx).Find(&logs).Error | |||
| if err != nil { | |||
| return nil, 0, err | |||
| } | |||
| @@ -343,7 +343,7 @@ func GetAllLogs(logType int, startTimestamp int64, endTimestamp int64, modelName | |||
| return logs, total, err | |||
| } | |||
| const logSearchCountLimit = 10000 | |||
| const logSearchCountLimit = 100000 | |||
| func GetUserLogs(userId int, logType int, startTimestamp int64, endTimestamp int64, modelName string, tokenName string, startIdx int, num int, group string, requestId string, chatId string, upstreamId string) (logs []*Log, total int64, err error) { | |||
| var tx *gorm.DB | |||
| @@ -386,7 +386,7 @@ func GetUserLogs(userId int, logType int, startTimestamp int64, endTimestamp int | |||
| common.SysError("failed to count user logs: " + err.Error()) | |||
| return nil, 0, errors.New("查询日志失败") | |||
| } | |||
| err = tx.Order("logs.id desc").Limit(num).Offset(startIdx).Find(&logs).Error | |||
| err = tx.Order("logs.created_at desc, logs.id desc").Limit(num).Offset(startIdx).Find(&logs).Error | |||
| if err != nil { | |||
| common.SysError("failed to search user logs: " + err.Error()) | |||
| return nil, 0, errors.New("查询日志失败") | |||
| @@ -207,9 +207,7 @@ func InitDB() (err error) { | |||
| return err | |||
| } | |||
| LoadEmailQuotaCache() | |||
| LoadChannelPricingCache() | |||
| LoadUserChannelRatioCache() | |||
| return nil | |||
| return nil | |||
| } else { | |||
| common.FatalLog(err) | |||
| } | |||
| @@ -284,17 +282,23 @@ func migrateDB() error { | |||
| &SubscriptionPreConsumeRecord{}, | |||
| &CustomOAuthProvider{}, | |||
| &UserOAuthBinding{}, | |||
| &ChannelPricing{}, | |||
| &PricingTag{}, | |||
| &UserChannelRatio{}, | |||
| &PendingSyncRecord{}, | |||
| &QuotaSyncLog{}, | |||
| &EmailQuotaRule{}, | |||
| &UserModelRateLimit{}, | |||
| &UserMigrationBatch{}, | |||
| &UserMigrationItem{}, | |||
| &MigrationQuotaGrant{}, | |||
| ) | |||
| if err != nil { | |||
| return err | |||
| } | |||
| if err := DB.Exec("DROP TABLE IF EXISTS channel_pricings").Error; err != nil { | |||
| return err | |||
| } | |||
| if err := DB.Exec("DROP TABLE IF EXISTS pricing_tags").Error; err != nil { | |||
| return err | |||
| } | |||
| if common.UsingSQLite { | |||
| if err := ensureSubscriptionPlanTableSQLite(); err != nil { | |||
| return err | |||
| @@ -303,14 +307,14 @@ func migrateDB() error { | |||
| if err := DB.AutoMigrate(&SubscriptionPlan{}); err != nil { | |||
| return err | |||
| } | |||
| } | |||
| // 将现有 sort_order=0 的模型和供应商更新为默认大数 | |||
| DB.Model(&Model{}).Where("sort_order = 0").Update("sort_order", 999999) | |||
| DB.Model(&Vendor{}).Where("sort_order = 0").Update("sort_order", 999999) | |||
| migrateChannelPublicName() | |||
| return nil | |||
| } | |||
| // 将现有 sort_order=0 的模型和供应商更新为默认大数 | |||
| DB.Model(&Model{}).Where("sort_order = 0").Update("sort_order", 999999) | |||
| DB.Model(&Vendor{}).Where("sort_order = 0").Update("sort_order", 999999) | |||
| migrateChannelPublicName() | |||
| return nil | |||
| } | |||
| func migrateDBFast() error { | |||
| // Drop bound_channel_id column from tokens table (deprecated field) | |||
| @@ -346,12 +350,12 @@ func migrateDBFast() error { | |||
| {&SubscriptionPreConsumeRecord{}, "SubscriptionPreConsumeRecord"}, | |||
| {&CustomOAuthProvider{}, "CustomOAuthProvider"}, | |||
| {&UserOAuthBinding{}, "UserOAuthBinding"}, | |||
| {&ChannelPricing{}, "ChannelPricing"}, | |||
| {&PricingTag{}, "PricingTag"}, | |||
| {&UserChannelRatio{}, "UserChannelRatio"}, | |||
| {&PendingSyncRecord{}, "PendingSyncRecord"}, | |||
| {&QuotaSyncLog{}, "QuotaSyncLog"}, | |||
| {&EmailQuotaRule{}, "EmailQuotaRule"}, | |||
| {&UserMigrationBatch{}, "UserMigrationBatch"}, | |||
| {&UserMigrationItem{}, "UserMigrationItem"}, | |||
| {&MigrationQuotaGrant{}, "MigrationQuotaGrant"}, | |||
| } | |||
| // 动态计算migration数量,确保errChan缓冲区足够大 | |||
| errChan := make(chan error, len(migrations)) | |||
| @@ -175,7 +175,7 @@ func InitOptionMap() { | |||
| } | |||
| common.OptionMapRWMutex.Unlock() | |||
| loadOptionsFromDatabase() | |||
| LoadOptionsFromDatabase() | |||
| } | |||
| // triggerWechatPayReset 安全触发微信支付客户端重置 | |||
| @@ -192,7 +192,7 @@ func triggerAlipayReset() { | |||
| } | |||
| } | |||
| func loadOptionsFromDatabase() { | |||
| func LoadOptionsFromDatabase() { | |||
| options, _ := AllOption() | |||
| for _, option := range options { | |||
| err := updateOptionMap(option.Key, option.Value) | |||
| @@ -206,7 +206,7 @@ func SyncOptions(frequency int) { | |||
| for { | |||
| time.Sleep(time.Duration(frequency) * time.Second) | |||
| common.SysLog("syncing options from database") | |||
| loadOptionsFromDatabase() | |||
| LoadOptionsFromDatabase() | |||
| } | |||
| } | |||
| @@ -567,7 +567,7 @@ func handleConfigUpdate(key, value string) bool { | |||
| configMap := map[string]string{ | |||
| configKey: value, | |||
| } | |||
| config.UpdateConfigFromMap(cfg, configMap) | |||
| _ = config.UpdateConfigFromMap(cfg, configMap) | |||
| // 特定配置的后处理 | |||
| if configName == "performance_setting" { | |||
| @@ -1,11 +1,9 @@ | |||
| package model | |||
| import ( | |||
| "encoding/json" | |||
| "fmt" | |||
| "sort" | |||
| "strings" | |||
| "sync" | |||
| "time" | |||
| @@ -32,7 +30,6 @@ type Pricing struct { | |||
| SupportedEndpointTypes []constant.EndpointType `json:"supported_endpoint_types"` | |||
| PricingVersion string `json:"pricing_version,omitempty"` | |||
| Type int `json:"type"` | |||
| DefaultChannelName string `json:"default_channel_name,omitempty"` | |||
| } | |||
| type PricingVendor struct { | |||
| @@ -42,6 +39,11 @@ type PricingVendor struct { | |||
| Icon string `json:"icon,omitempty"` | |||
| } | |||
| const ( | |||
| QuotaTypeByTokens = 0 | |||
| QuotaTypeByCall = 1 | |||
| ) | |||
| var ( | |||
| pricingMap []Pricing | |||
| pricingByModel map[string]*Pricing | |||
| @@ -50,7 +52,6 @@ var ( | |||
| lastGetPricingTime time.Time | |||
| updatePricingLock sync.Mutex | |||
| // 缓存映射:模型名 -> 启用分组 / 计费类型 | |||
| modelEnableGroups = make(map[string][]string) | |||
| modelQuotaTypeMap = make(map[string]int) | |||
| modelEnableGroupsLock = sync.RWMutex{} | |||
| @@ -61,7 +62,6 @@ var ( | |||
| modelSupportEndpointsLock = sync.RWMutex{} | |||
| ) | |||
| // SetTestPricing 设置测试用的定价缓存(仅用于测试) | |||
| func SetTestPricing(pricing []Pricing) { | |||
| updatePricingLock.Lock() | |||
| defer updatePricingLock.Unlock() | |||
| @@ -75,11 +75,10 @@ func SetTestPricing(pricing []Pricing) { | |||
| } | |||
| func GetPricing() []Pricing { | |||
| if time.Since(lastGetPricingTime) > time.Minute*1 || len(pricingMap) == 0 { | |||
| if time.Since(lastGetPricingTime) > time.Minute || len(pricingMap) == 0 { | |||
| updatePricingLock.Lock() | |||
| defer updatePricingLock.Unlock() | |||
| // Double check after acquiring the lock | |||
| if time.Since(lastGetPricingTime) > time.Minute*1 || len(pricingMap) == 0 { | |||
| if time.Since(lastGetPricingTime) > time.Minute || len(pricingMap) == 0 { | |||
| modelSupportEndpointsLock.Lock() | |||
| defer modelSupportEndpointsLock.Unlock() | |||
| updatePricing() | |||
| @@ -88,19 +87,16 @@ func GetPricing() []Pricing { | |||
| return pricingMap | |||
| } | |||
| // GetPricingByModel 从缓存中查找指定模型的定价信息 | |||
| func GetPricingByModel(modelName string) *Pricing { | |||
| GetPricing() // 确保缓存已刷新 | |||
| GetPricing() | |||
| modelEnableGroupsLock.RLock() | |||
| p := pricingByModel[modelName] | |||
| modelEnableGroupsLock.RUnlock() | |||
| return p | |||
| } | |||
| // GetVendors 返回当前定价接口使用到的供应商信息 | |||
| func GetVendors() []PricingVendor { | |||
| if time.Since(lastGetPricingTime) > time.Minute*1 || len(pricingMap) == 0 { | |||
| // 保证先刷新一次 | |||
| if time.Since(lastGetPricingTime) > time.Minute || len(pricingMap) == 0 { | |||
| GetPricing() | |||
| } | |||
| return vendorsList | |||
| @@ -118,14 +114,55 @@ func GetModelSupportEndpointTypes(model string) []constant.EndpointType { | |||
| return make([]constant.EndpointType, 0) | |||
| } | |||
| func parseModelEndpointTypes(rawValue string) []string { | |||
| trimmed := strings.TrimSpace(rawValue) | |||
| if trimmed == "" { | |||
| return nil | |||
| } | |||
| var rawObject map[string]interface{} | |||
| if err := common.Unmarshal([]byte(trimmed), &rawObject); err == nil { | |||
| endpoints := make([]string, 0, len(rawObject)) | |||
| for k, v := range rawObject { | |||
| switch v.(type) { | |||
| case string, map[string]interface{}: | |||
| if !common.StringsContains(endpoints, k) { | |||
| endpoints = append(endpoints, k) | |||
| } | |||
| } | |||
| } | |||
| if len(endpoints) == 0 { | |||
| return nil | |||
| } | |||
| return endpoints | |||
| } | |||
| var rawList []string | |||
| if err := common.Unmarshal([]byte(trimmed), &rawList); err == nil { | |||
| endpoints := make([]string, 0, len(rawList)) | |||
| for _, endpoint := range rawList { | |||
| endpoint = strings.TrimSpace(endpoint) | |||
| if endpoint == "" || common.StringsContains(endpoints, endpoint) { | |||
| continue | |||
| } | |||
| endpoints = append(endpoints, endpoint) | |||
| } | |||
| if len(endpoints) == 0 { | |||
| return nil | |||
| } | |||
| return endpoints | |||
| } | |||
| return nil | |||
| } | |||
| func updatePricing() { | |||
| //modelRatios := common.GetModelRatios() | |||
| enableAbilities, err := GetAllEnableAbilityWithChannels() | |||
| if err != nil { | |||
| common.SysLog(fmt.Sprintf("GetAllEnableAbilityWithChannels error: %v", err)) | |||
| return | |||
| } | |||
| // 预加载模型元数据与供应商一次,避免循环查询 | |||
| var allMeta []Model | |||
| _ = DB.Find(&allMeta).Error | |||
| metaMap := make(map[string]*Model) | |||
| @@ -136,60 +173,56 @@ func updatePricing() { | |||
| m := &allMeta[i] | |||
| if m.NameRule == NameRuleExact { | |||
| metaMap[m.ModelName] = m | |||
| } else { | |||
| switch m.NameRule { | |||
| case NameRulePrefix: | |||
| prefixList = append(prefixList, m) | |||
| case NameRuleSuffix: | |||
| suffixList = append(suffixList, m) | |||
| case NameRuleContains: | |||
| containsList = append(containsList, m) | |||
| } | |||
| continue | |||
| } | |||
| switch m.NameRule { | |||
| case NameRulePrefix: | |||
| prefixList = append(prefixList, m) | |||
| case NameRuleSuffix: | |||
| suffixList = append(suffixList, m) | |||
| case NameRuleContains: | |||
| containsList = append(containsList, m) | |||
| } | |||
| } | |||
| // 将非精确规则模型匹配到 metaMap | |||
| for _, m := range prefixList { | |||
| for _, pricingModel := range enableAbilities { | |||
| if strings.HasPrefix(pricingModel.Model, m.ModelName) { | |||
| if _, exists := metaMap[pricingModel.Model]; !exists { | |||
| metaMap[pricingModel.Model] = m | |||
| for _, ability := range enableAbilities { | |||
| if strings.HasPrefix(ability.Model, m.ModelName) { | |||
| if _, exists := metaMap[ability.Model]; !exists { | |||
| metaMap[ability.Model] = m | |||
| } | |||
| } | |||
| } | |||
| } | |||
| for _, m := range suffixList { | |||
| for _, pricingModel := range enableAbilities { | |||
| if strings.HasSuffix(pricingModel.Model, m.ModelName) { | |||
| if _, exists := metaMap[pricingModel.Model]; !exists { | |||
| metaMap[pricingModel.Model] = m | |||
| for _, ability := range enableAbilities { | |||
| if strings.HasSuffix(ability.Model, m.ModelName) { | |||
| if _, exists := metaMap[ability.Model]; !exists { | |||
| metaMap[ability.Model] = m | |||
| } | |||
| } | |||
| } | |||
| } | |||
| for _, m := range containsList { | |||
| for _, pricingModel := range enableAbilities { | |||
| if strings.Contains(pricingModel.Model, m.ModelName) { | |||
| if _, exists := metaMap[pricingModel.Model]; !exists { | |||
| metaMap[pricingModel.Model] = m | |||
| for _, ability := range enableAbilities { | |||
| if strings.Contains(ability.Model, m.ModelName) { | |||
| if _, exists := metaMap[ability.Model]; !exists { | |||
| metaMap[ability.Model] = m | |||
| } | |||
| } | |||
| } | |||
| } | |||
| // 预加载供应商 | |||
| var vendors []Vendor | |||
| _ = DB.Find(&vendors).Error | |||
| vendorMap := make(map[int]*Vendor) | |||
| vendorMap := make(map[int]*Vendor, len(vendors)) | |||
| for i := range vendors { | |||
| vendorMap[vendors[i].Id] = &vendors[i] | |||
| } | |||
| // 初始化默认供应商映射 | |||
| initDefaultVendorMapping(metaMap, vendorMap, enableAbilities) | |||
| // 构建对前端友好的供应商列表 | |||
| vendorOrderMap := make(map[int]int) | |||
| vendorOrderMap := make(map[int]int, len(vendorMap)) | |||
| for _, v := range vendorMap { | |||
| vendorOrderMap[v.Id] = v.SortOrder | |||
| } | |||
| @@ -213,7 +246,6 @@ func updatePricing() { | |||
| }) | |||
| modelGroupsMap := make(map[string]*types.Set[string]) | |||
| for _, ability := range enableAbilities { | |||
| groups, ok := modelGroupsMap[ability.Model] | |||
| if !ok { | |||
| @@ -223,10 +255,7 @@ func updatePricing() { | |||
| groups.Add(ability.Group) | |||
| } | |||
| //这里使用切片而不是Set,因为一个模型可能支持多个端点类型,并且第一个端点是优先使用端点 | |||
| modelSupportEndpointsStr := make(map[string][]string) | |||
| // 先根据已有能力填充原生端点 | |||
| for _, ability := range enableAbilities { | |||
| endpoints := modelSupportEndpointsStr[ability.Model] | |||
| channelTypes := common.GetEndpointTypesByChannelType(ability.ChannelType, ability.Model) | |||
| @@ -238,41 +267,25 @@ func updatePricing() { | |||
| modelSupportEndpointsStr[ability.Model] = endpoints | |||
| } | |||
| // 再补充模型自定义端点:若配置有效则替换默认端点,不做合并 | |||
| for modelName, meta := range metaMap { | |||
| if strings.TrimSpace(meta.Endpoints) == "" { | |||
| continue | |||
| } | |||
| var raw map[string]interface{} | |||
| if err := json.Unmarshal([]byte(meta.Endpoints), &raw); err == nil { | |||
| endpoints := make([]string, 0, len(raw)) | |||
| for k, v := range raw { | |||
| switch v.(type) { | |||
| case string, map[string]interface{}: | |||
| if !common.StringsContains(endpoints, k) { | |||
| endpoints = append(endpoints, k) | |||
| } | |||
| } | |||
| } | |||
| if len(endpoints) > 0 { | |||
| modelSupportEndpointsStr[modelName] = endpoints | |||
| } | |||
| if endpoints := parseModelEndpointTypes(meta.Endpoints); len(endpoints) > 0 { | |||
| modelSupportEndpointsStr[modelName] = endpoints | |||
| } | |||
| } | |||
| modelSupportEndpointTypes = make(map[string][]constant.EndpointType) | |||
| for model, endpoints := range modelSupportEndpointsStr { | |||
| supportedEndpoints := make([]constant.EndpointType, 0) | |||
| supportedEndpoints := make([]constant.EndpointType, 0, len(endpoints)) | |||
| for _, endpointStr := range endpoints { | |||
| endpointType := constant.EndpointType(endpointStr) | |||
| supportedEndpoints = append(supportedEndpoints, endpointType) | |||
| supportedEndpoints = append(supportedEndpoints, constant.EndpointType(endpointStr)) | |||
| } | |||
| modelSupportEndpointTypes[model] = supportedEndpoints | |||
| } | |||
| // 构建全局 supportedEndpointMap(默认 + 自定义覆盖) | |||
| supportedEndpointMap = make(map[string]common.EndpointInfo) | |||
| // 1. 默认端点 | |||
| for _, endpoints := range modelSupportEndpointTypes { | |||
| for _, et := range endpoints { | |||
| if info, ok := common.GetDefaultEndpointInfo(et); ok { | |||
| @@ -282,13 +295,12 @@ func updatePricing() { | |||
| } | |||
| } | |||
| } | |||
| // 2. 自定义端点(models 表)覆盖默认 | |||
| for _, meta := range metaMap { | |||
| if strings.TrimSpace(meta.Endpoints) == "" { | |||
| continue | |||
| } | |||
| var raw map[string]interface{} | |||
| if err := json.Unmarshal([]byte(meta.Endpoints), &raw); err == nil { | |||
| if err := common.Unmarshal([]byte(meta.Endpoints), &raw); err == nil { | |||
| for k, v := range raw { | |||
| switch val := v.(type) { | |||
| case string: | |||
| @@ -302,32 +314,12 @@ func updatePricing() { | |||
| ep.Method = strings.ToUpper(m) | |||
| } | |||
| supportedEndpointMap[k] = ep | |||
| default: | |||
| // ignore unsupported types | |||
| } | |||
| } | |||
| } | |||
| } | |||
| // 从渠道定价表加载实际定价数据(仅启用渠道),同时获取渠道名称 | |||
| var allCPs []struct { | |||
| ChannelPricing | |||
| ChannelName string | |||
| ChannelPublicName string | |||
| } | |||
| DB.Table("channel_pricings"). | |||
| Select("channel_pricings.*, channels.name as channel_name, channels.public_name as channel_public_name"). | |||
| Joins("JOIN channels ON channel_pricings.channel_id = channels.id"). | |||
| Where("channels.status = 1 AND channel_pricings.deleted_at IS NULL"). | |||
| Find(&allCPs) | |||
| cpMap := make(map[string][]ChannelPricing) | |||
| channelNameMap := make(map[int]string) | |||
| for i := range allCPs { | |||
| cpMap[allCPs[i].ModelName] = append(cpMap[allCPs[i].ModelName], allCPs[i].ChannelPricing) | |||
| channelNameMap[allCPs[i].ChannelId] = ChannelDisplayName(allCPs[i].ChannelPublicName, allCPs[i].ChannelName) | |||
| } | |||
| pricingMap = make([]Pricing, 0) | |||
| pricingMap = make([]Pricing, 0, len(modelGroupsMap)) | |||
| for model, groups := range modelGroupsMap { | |||
| pricing := Pricing{ | |||
| ModelName: model, | |||
| @@ -335,9 +327,7 @@ func updatePricing() { | |||
| SupportedEndpointTypes: modelSupportEndpointTypes[model], | |||
| } | |||
| // 补充模型元数据(描述、标签、供应商、状态) | |||
| if meta, ok := metaMap[model]; ok { | |||
| // 若模型被禁用(status==0),则直接跳过,不返回给前端 | |||
| if meta.Status == 0 { | |||
| continue | |||
| } | |||
| @@ -348,19 +338,10 @@ func updatePricing() { | |||
| pricing.Type = meta.Type | |||
| } | |||
| // 使用渠道定价表中的实际数据,选取最便宜的渠道 | |||
| applyBestChannelPricing(&pricing, cpMap[model], model) | |||
| // 填充默认通道名称 | |||
| if chId, ok := GetDefaultChannelId(model); ok { | |||
| if name, found := channelNameMap[chId]; found { | |||
| pricing.DefaultChannelName = name | |||
| } | |||
| } | |||
| applyGlobalDefault(&pricing, model) | |||
| pricingMap = append(pricingMap, pricing) | |||
| } | |||
| // 按 sort_order 排序 pricingMap,999999 视为未设置 | |||
| sort.Slice(pricingMap, func(i, j int) bool { | |||
| mi, okI := metaMap[pricingMap[i].ModelName] | |||
| mj, okJ := metaMap[pricingMap[j].ModelName] | |||
| @@ -378,12 +359,10 @@ func updatePricing() { | |||
| return pricingMap[i].ModelName < pricingMap[j].ModelName | |||
| }) | |||
| // 防止大更新后数据不通用 | |||
| if len(pricingMap) > 0 { | |||
| pricingMap[0].PricingVersion = "82c4a357505fff6fee8462c3f7ec8a645bb95532669cb73b2cabee6a416ec24f" | |||
| } | |||
| // 刷新缓存映射,供高并发快速查询 | |||
| modelEnableGroupsLock.Lock() | |||
| modelEnableGroups = make(map[string][]string) | |||
| modelQuotaTypeMap = make(map[string]int) | |||
| @@ -400,79 +379,24 @@ func updatePricing() { | |||
| lastGetPricingTime = time.Now() | |||
| } | |||
| // GetSupportedEndpointMap 返回全局端点到路径的映射 | |||
| func GetSupportedEndpointMap() map[string]common.EndpointInfo { | |||
| return supportedEndpointMap | |||
| } | |||
| // applyGlobalDefault 用全局默认值填充 Pricing(无渠道定价时的回退) | |||
| func applyGlobalDefault(pricing *Pricing, model string) { | |||
| modelPrice, findPrice := ratio_setting.GetModelPrice(model, false) | |||
| if findPrice { | |||
| pricing.ModelPrice = modelPrice | |||
| pricing.QuotaType = 1 | |||
| pricing.ModelRatio = 0 | |||
| pricing.CompletionRatio = 0 | |||
| pricing.QuotaType = QuotaTypeByCall | |||
| } else { | |||
| modelRatio, _, _ := ratio_setting.GetModelRatio(model) | |||
| pricing.ModelPrice = 0 | |||
| pricing.ModelRatio = modelRatio | |||
| pricing.CompletionRatio = ratio_setting.GetCompletionRatio(model) | |||
| pricing.QuotaType = 0 | |||
| pricing.QuotaType = QuotaTypeByTokens | |||
| } | |||
| pricing.CacheRatio, _ = ratio_setting.GetCacheRatio(model) | |||
| pricing.CacheCreationRatio, _ = ratio_setting.GetCreateCacheRatio(model) | |||
| } | |||
| // applyBestChannelPricing 从渠道定价中选取最优(最便宜)的价格填充 Pricing | |||
| // 优先选按量计费(quota_type=0)的渠道,因为缓存价格仅对按量计费有意义 | |||
| // 扩展比率字段(cache_ratio 等)为 0 表示未设置,需回退到全局默认值 | |||
| func applyBestChannelPricing(pricing *Pricing, cps []ChannelPricing, model string) { | |||
| if len(cps) == 0 { | |||
| applyGlobalDefault(pricing, model) | |||
| return | |||
| } | |||
| // 优先选按量计费 (quota_type=0) 中 model_ratio 最低的渠道 | |||
| var bestPerToken *ChannelPricing | |||
| for i := range cps { | |||
| cp := &cps[i] | |||
| if cp.QuotaType == 0 { | |||
| if bestPerToken == nil || cp.ModelRatio < bestPerToken.ModelRatio { | |||
| bestPerToken = cp | |||
| } | |||
| } | |||
| } | |||
| if bestPerToken != nil { | |||
| pricing.QuotaType = 0 | |||
| pricing.ModelRatio = bestPerToken.ModelRatio | |||
| pricing.CompletionRatio = bestPerToken.CompletionRatio | |||
| if bestPerToken.CacheRatio > 0 { | |||
| pricing.CacheRatio = bestPerToken.CacheRatio | |||
| } else { | |||
| pricing.CacheRatio, _ = ratio_setting.GetCacheRatio(model) | |||
| } | |||
| if bestPerToken.CacheCreationRatio > 0 { | |||
| pricing.CacheCreationRatio = bestPerToken.CacheCreationRatio | |||
| } else { | |||
| pricing.CacheCreationRatio, _ = ratio_setting.GetCreateCacheRatio(model) | |||
| } | |||
| return | |||
| } | |||
| // 没有按量渠道,选按次计费 (quota_type=1) 中 model_price 最低的渠道 | |||
| var bestPerCall *ChannelPricing | |||
| for i := range cps { | |||
| cp := &cps[i] | |||
| if cp.QuotaType == 1 { | |||
| if bestPerCall == nil || cp.ModelPrice < bestPerCall.ModelPrice { | |||
| bestPerCall = cp | |||
| } | |||
| } | |||
| } | |||
| if bestPerCall != nil { | |||
| pricing.QuotaType = 1 | |||
| pricing.ModelPrice = bestPerCall.ModelPrice | |||
| return | |||
| } | |||
| applyGlobalDefault(pricing, model) | |||
| } | |||
| @@ -1,60 +0,0 @@ | |||
| package model | |||
| import ( | |||
| "github.com/QuantumNous/new-api/common" | |||
| "gorm.io/gorm" | |||
| ) | |||
| // PricingTag 定价标签表 | |||
| // 用于对渠道定价进行分类和标识 | |||
| type PricingTag struct { | |||
| Id int `json:"id" gorm:"primaryKey"` | |||
| Name string `json:"name" gorm:"size:64;not null;uniqueIndex"` | |||
| Color string `json:"color" gorm:"size:16;default:'#1890ff'"` | |||
| Description string `json:"description" gorm:"type:text"` | |||
| SortOrder int `json:"sort_order" gorm:"default:0"` | |||
| CreatedTime int64 `json:"created_time" gorm:"bigint"` | |||
| DeletedAt gorm.DeletedAt `json:"-" gorm:"index"` | |||
| } | |||
| func (pt *PricingTag) Insert() error { | |||
| pt.CreatedTime = common.GetTimestamp() | |||
| return DB.Create(pt).Error | |||
| } | |||
| func (pt *PricingTag) Update() error { | |||
| return DB.Model(&PricingTag{}).Where("id = ?", pt.Id).Updates(map[string]interface{}{ | |||
| "name": pt.Name, | |||
| "color": pt.Color, | |||
| "description": pt.Description, | |||
| "sort_order": pt.SortOrder, | |||
| }).Error | |||
| } | |||
| func (pt *PricingTag) Delete() error { | |||
| return DB.Delete(pt).Error | |||
| } | |||
| func GetPricingTagById(id int) (*PricingTag, error) { | |||
| var pt PricingTag | |||
| err := DB.First(&pt, id).Error | |||
| if err != nil { | |||
| return nil, err | |||
| } | |||
| return &pt, nil | |||
| } | |||
| func GetAllPricingTags() ([]*PricingTag, error) { | |||
| var list []*PricingTag | |||
| err := DB.Order("sort_order ASC, id ASC").Find(&list).Error | |||
| return list, err | |||
| } | |||
| func GetPricingTagByName(name string) (*PricingTag, error) { | |||
| var pt PricingTag | |||
| err := DB.Where("name = ?", name).First(&pt).Error | |||
| if err != nil { | |||
| return nil, err | |||
| } | |||
| return &pt, nil | |||
| } | |||
| @@ -2,121 +2,163 @@ package model | |||
| import ( | |||
| "testing" | |||
| "time" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/constant" | |||
| "github.com/QuantumNous/new-api/setting/ratio_setting" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| const testPricingModel = "test-pricing-model-apply" | |||
| const testPricingApplyModel = "test-pricing-model-apply" | |||
| const testPricingCallModel = "test-pricing-call-model" | |||
| const testPricingGlobalModel = "test-pricing-global-model" | |||
| func resetPricingRatios(t *testing.T) { | |||
| t.Helper() | |||
| require.NoError(t, ratio_setting.UpdateModelRatioByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateCompletionRatioByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateCacheRatioByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateCreateCacheRatioByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateModelPriceByJSONString(`{}`)) | |||
| } | |||
| func setupPricingTest(t *testing.T) { | |||
| t.Helper() | |||
| resetPricingRatios(t) | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, _ := db.DB() | |||
| sqlDB, err := db.DB() | |||
| require.NoError(t, err) | |||
| sqlDB.SetMaxOpenConns(1) | |||
| origDB := DB | |||
| origUsingSQLite := common.UsingSQLite | |||
| origPricingMap := pricingMap | |||
| origPricingByModel := pricingByModel | |||
| origVendorsList := vendorsList | |||
| origSupportedEndpointMap := supportedEndpointMap | |||
| origModelSupportEndpointTypes := modelSupportEndpointTypes | |||
| origLastGetPricingTime := lastGetPricingTime | |||
| DB = db | |||
| require.NoError(t, db.AutoMigrate(&ChannelPricing{})) | |||
| common.UsingSQLite = true | |||
| pricingMap = nil | |||
| pricingByModel = nil | |||
| vendorsList = nil | |||
| supportedEndpointMap = nil | |||
| modelSupportEndpointTypes = make(map[string][]constant.EndpointType) | |||
| lastGetPricingTime = time.Time{} | |||
| // 全局默认定价 | |||
| require.NoError(t, ratio_setting.UpdateModelRatioByJSONString(`{"`+testPricingModel+`":10}`)) | |||
| require.NoError(t, ratio_setting.UpdateCompletionRatioByJSONString(`{"`+testPricingModel+`":3}`)) | |||
| require.NoError(t, ratio_setting.UpdateCacheRatioByJSONString(`{"`+testPricingModel+`":0.5}`)) | |||
| require.NoError(t, ratio_setting.UpdateCreateCacheRatioByJSONString(`{"`+testPricingModel+`":0.75}`)) | |||
| require.NoError(t, db.AutoMigrate(&Model{}, &Vendor{}, &Channel{}, &Ability{})) | |||
| t.Cleanup(func() { | |||
| DB = origDB | |||
| sqlDB.Close() | |||
| ratio_setting.UpdateModelRatioByJSONString(`{}`) | |||
| ratio_setting.UpdateCompletionRatioByJSONString(`{}`) | |||
| ratio_setting.UpdateCacheRatioByJSONString(`{}`) | |||
| ratio_setting.UpdateCreateCacheRatioByJSONString(`{}`) | |||
| ratio_setting.UpdateModelPriceByJSONString(`{}`) | |||
| common.UsingSQLite = origUsingSQLite | |||
| pricingMap = origPricingMap | |||
| pricingByModel = origPricingByModel | |||
| vendorsList = origVendorsList | |||
| supportedEndpointMap = origSupportedEndpointMap | |||
| modelSupportEndpointTypes = origModelSupportEndpointTypes | |||
| lastGetPricingTime = origLastGetPricingTime | |||
| require.NoError(t, sqlDB.Close()) | |||
| resetPricingRatios(t) | |||
| }) | |||
| } | |||
| func TestApplyBestChannelPricing_NoChannelPricing(t *testing.T) { | |||
| setupPricingTest(t) | |||
| func TestApplyGlobalDefault_PerTokenModel(t *testing.T) { | |||
| resetPricingRatios(t) | |||
| t.Cleanup(func() { resetPricingRatios(t) }) | |||
| require.NoError(t, ratio_setting.UpdateModelRatioByJSONString(`{"`+testPricingApplyModel+`":10}`)) | |||
| require.NoError(t, ratio_setting.UpdateCompletionRatioByJSONString(`{"`+testPricingApplyModel+`":3}`)) | |||
| require.NoError(t, ratio_setting.UpdateCacheRatioByJSONString(`{"`+testPricingApplyModel+`":0.5}`)) | |||
| require.NoError(t, ratio_setting.UpdateCreateCacheRatioByJSONString(`{"`+testPricingApplyModel+`":0.75}`)) | |||
| p := &Pricing{} | |||
| applyBestChannelPricing(p, nil, testPricingModel) | |||
| applyGlobalDefault(p, testPricingApplyModel) | |||
| require.Equal(t, 0, p.QuotaType, "应使用全局按量计费") | |||
| require.Equal(t, 10.0, p.ModelRatio, "应使用全局 model_ratio") | |||
| require.Equal(t, 3.0, p.CompletionRatio, "应使用全局 completion_ratio") | |||
| require.Equal(t, 0.5, p.CacheRatio, "应使用全局 cache_ratio") | |||
| require.Equal(t, 0.75, p.CacheCreationRatio, "应使用全局 cache_creation_ratio") | |||
| require.Equal(t, QuotaTypeByTokens, p.QuotaType) | |||
| require.Equal(t, 10.0, p.ModelRatio) | |||
| require.Equal(t, 3.0, p.CompletionRatio) | |||
| require.Equal(t, 0.5, p.CacheRatio) | |||
| require.Equal(t, 0.75, p.CacheCreationRatio) | |||
| } | |||
| func TestApplyBestChannelPricing_PerTokenCheapest(t *testing.T) { | |||
| setupPricingTest(t) | |||
| // 渠道A: model_ratio=8(更便宜) | |||
| require.NoError(t, (&ChannelPricing{ | |||
| ModelName: testPricingModel, ChannelId: 1, | |||
| QuotaType: QuotaTypeByTokens, ModelRatio: 8, CompletionRatio: 2, | |||
| CacheRatio: 0.3, CacheCreationRatio: 0.6, | |||
| }).Insert()) | |||
| // 渠道B: model_ratio=12(更贵) | |||
| require.NoError(t, (&ChannelPricing{ | |||
| ModelName: testPricingModel, ChannelId: 2, | |||
| QuotaType: QuotaTypeByTokens, ModelRatio: 12, CompletionRatio: 4, | |||
| CacheRatio: 0.8, CacheCreationRatio: 1.0, | |||
| }).Insert()) | |||
| func TestApplyGlobalDefault_PerCallModel(t *testing.T) { | |||
| resetPricingRatios(t) | |||
| t.Cleanup(func() { resetPricingRatios(t) }) | |||
| cps := []ChannelPricing{ | |||
| {ModelRatio: 8, CompletionRatio: 2, CacheRatio: 0.3, CacheCreationRatio: 0.6}, | |||
| {ModelRatio: 12, CompletionRatio: 4, CacheRatio: 0.8, CacheCreationRatio: 1.0}, | |||
| } | |||
| require.NoError(t, ratio_setting.UpdateModelPriceByJSONString(`{"`+testPricingCallModel+`":0.3}`)) | |||
| p := &Pricing{} | |||
| applyBestChannelPricing(p, cps, testPricingModel) | |||
| applyGlobalDefault(p, testPricingCallModel) | |||
| require.Equal(t, 0, p.QuotaType) | |||
| require.Equal(t, 8.0, p.ModelRatio, "应选最便宜的渠道A") | |||
| require.Equal(t, 2.0, p.CompletionRatio) | |||
| require.Equal(t, 0.3, p.CacheRatio) | |||
| require.Equal(t, 0.6, p.CacheCreationRatio) | |||
| require.Equal(t, QuotaTypeByCall, p.QuotaType) | |||
| require.Equal(t, 0.3, p.ModelPrice) | |||
| require.Equal(t, 0.0, p.ModelRatio) | |||
| } | |||
| func TestApplyBestChannelPricing_ExtendedRatioZeroFallback(t *testing.T) { | |||
| func TestGetPricingByModel_UsesGlobalDefaultOnly(t *testing.T) { | |||
| setupPricingTest(t) | |||
| // 渠道定价中扩展比率为 0,应回退到全局值 | |||
| require.NoError(t, (&ChannelPricing{ | |||
| ModelName: testPricingModel, ChannelId: 1, | |||
| QuotaType: QuotaTypeByTokens, ModelRatio: 5, CompletionRatio: 1, | |||
| CacheRatio: 0, CacheCreationRatio: 0, // 0 = 未设置 | |||
| }).Insert()) | |||
| require.NoError(t, ratio_setting.UpdateModelRatioByJSONString(`{"`+testPricingGlobalModel+`":10}`)) | |||
| require.NoError(t, ratio_setting.UpdateCompletionRatioByJSONString(`{"`+testPricingGlobalModel+`":3}`)) | |||
| require.NoError(t, ratio_setting.UpdateCacheRatioByJSONString(`{"`+testPricingGlobalModel+`":0.5}`)) | |||
| require.NoError(t, ratio_setting.UpdateCreateCacheRatioByJSONString(`{"`+testPricingGlobalModel+`":0.75}`)) | |||
| cps := []ChannelPricing{ | |||
| {ModelRatio: 5, CompletionRatio: 1, CacheRatio: 0, CacheCreationRatio: 0}, | |||
| } | |||
| require.NoError(t, (&Model{ | |||
| ModelName: testPricingGlobalModel, | |||
| Status: 1, | |||
| }).Insert()) | |||
| require.NoError(t, (&Vendor{ | |||
| Id: 1, | |||
| Name: "test-vendor", | |||
| SortOrder: 1, | |||
| }).Insert()) | |||
| require.NoError(t, (&Channel{ | |||
| Id: 1, | |||
| Type: constant.ChannelTypeOpenAI, | |||
| Key: "k", | |||
| Name: "channel-a", | |||
| Status: common.ChannelStatusEnabled, | |||
| Models: testPricingGlobalModel, | |||
| Group: "default", | |||
| }).Insert()) | |||
| p := &Pricing{} | |||
| applyBestChannelPricing(p, cps, testPricingModel) | |||
| RefreshPricing() | |||
| pricing := GetPricingByModel(testPricingGlobalModel) | |||
| require.Equal(t, 0.5, p.CacheRatio, "cache_ratio=0 应回退全局 0.5") | |||
| require.Equal(t, 0.75, p.CacheCreationRatio, "cache_creation_ratio=0 应回退全局 0.75") | |||
| require.NotNil(t, pricing) | |||
| require.Equal(t, QuotaTypeByTokens, pricing.QuotaType) | |||
| require.Equal(t, 10.0, pricing.ModelRatio) | |||
| require.Equal(t, 3.0, pricing.CompletionRatio) | |||
| require.Equal(t, 0.5, pricing.CacheRatio) | |||
| require.Equal(t, 0.75, pricing.CacheCreationRatio) | |||
| } | |||
| func TestApplyBestChannelPricing_PerCallCheapest(t *testing.T) { | |||
| setupPricingTest(t) | |||
| require.NoError(t, ratio_setting.UpdateModelPriceByJSONString(`{"`+testPricingModel+`":0.8}`)) | |||
| func TestParseModelEndpointTypes_SupportsObjectAndArray(t *testing.T) { | |||
| objectEndpoints := parseModelEndpointTypes(`{ | |||
| "anthropic": {"path": "/v1/messages", "method": "POST"}, | |||
| "openai": "/v1/chat/completions" | |||
| }`) | |||
| require.ElementsMatch(t, []string{"anthropic", "openai"}, objectEndpoints) | |||
| // 只有按次计费的渠道 | |||
| cps := []ChannelPricing{ | |||
| {QuotaType: QuotaTypeByCall, ModelPrice: 0.3}, | |||
| {QuotaType: QuotaTypeByCall, ModelPrice: 0.5}, | |||
| } | |||
| arrayEndpoints := parseModelEndpointTypes(`["anthropic", "openai", "anthropic"]`) | |||
| require.Equal(t, []string{"anthropic", "openai"}, arrayEndpoints) | |||
| } | |||
| p := &Pricing{} | |||
| applyBestChannelPricing(p, cps, testPricingModel) | |||
| func TestParseModelEndpointTypes_IgnoresInvalidJSON(t *testing.T) { | |||
| require.Nil(t, parseModelEndpointTypes(`123`)) | |||
| require.Nil(t, parseModelEndpointTypes(`{"anthropic": true}`)) | |||
| require.Nil(t, parseModelEndpointTypes(``)) | |||
| require.Equal(t, 1, p.QuotaType) | |||
| require.Equal(t, 0.3, p.ModelPrice, "应选最便宜的按次渠道") | |||
| info, ok := common.GetDefaultEndpointInfo("anthropic") | |||
| require.True(t, ok) | |||
| require.Equal(t, "/v1/messages", info.Path) | |||
| } | |||
| @@ -14,6 +14,9 @@ import ( | |||
| // ErrRedeemFailed is returned when redemption fails due to database error | |||
| var ErrRedeemFailed = errors.New("redeem.failed") | |||
| // ErrSyncedUserRedeemDenied is returned when a synced user tries to redeem a code. | |||
| var ErrSyncedUserRedeemDenied = errors.New("redemption.synced_user_denied") | |||
| type Redemption struct { | |||
| Id int `json:"id"` | |||
| UserId int `json:"user_id"` | |||
| @@ -123,6 +126,15 @@ func Redeem(key string, userId int) (quota int, err error) { | |||
| if userId == 0 { | |||
| return 0, errors.New("无效的 user id") | |||
| } | |||
| user := &User{} | |||
| err = DB.Select("id", "source").First(user, "id = ?", userId).Error | |||
| if err != nil { | |||
| return 0, err | |||
| } | |||
| if user.IsSyncedUser() { | |||
| return 0, ErrSyncedUserRedeemDenied | |||
| } | |||
| redemption := &Redemption{} | |||
| keyCol := "`key`" | |||
| @@ -241,15 +241,54 @@ func TestRedeem_SyncedUser(t *testing.T) { | |||
| // 执行兑换 | |||
| quota, err := Redeem("synced-redeem-key", 100) | |||
| require.NoError(t, err) | |||
| assert.Equal(t, 50000, quota) | |||
| require.ErrorIs(t, err, ErrSyncedUserRedeemDenied) | |||
| assert.Equal(t, 0, quota) | |||
| // 验证回调被触发(Master 节点需要同步到 Slave) | |||
| assert.True(t, callbackCalled) | |||
| // 同步用户不能兑换,回调不应触发 | |||
| assert.False(t, callbackCalled) | |||
| // 验证用户额度已更新 | |||
| // 验证用户额度和同步额度均未更新 | |||
| var updatedUser User | |||
| require.NoError(t, db.First(&updatedUser, 100).Error) | |||
| assert.Equal(t, 100000, updatedUser.Quota) | |||
| assert.Equal(t, 100000, updatedUser.SyncedQuota) | |||
| // 验证兑换码仍保持可用,不能被同步用户消耗 | |||
| var updatedRedemption Redemption | |||
| require.NoError(t, db.First(&updatedRedemption, 1).Error) | |||
| assert.Equal(t, common.RedemptionCodeStatusEnabled, updatedRedemption.Status) | |||
| assert.Equal(t, 0, updatedRedemption.UsedUserId) | |||
| assert.Equal(t, int64(0), updatedRedemption.RedeemedTime) | |||
| } | |||
| func TestRedeem_LocalUserStillWorks(t *testing.T) { | |||
| db := setupRedemptionDB(t) | |||
| user := User{ | |||
| Id: 101, | |||
| Username: "local_redeem_test", | |||
| Password: "hashed_password", | |||
| Quota: 100000, | |||
| Source: common.UserSourceLocal, | |||
| } | |||
| require.NoError(t, db.Create(&user).Error) | |||
| redemption := Redemption{ | |||
| Id: 2, | |||
| UserId: 1, | |||
| Key: "local-redeem-key", | |||
| Status: common.RedemptionCodeStatusEnabled, | |||
| Quota: 50000, | |||
| CreatedTime: common.GetTimestamp(), | |||
| } | |||
| require.NoError(t, db.Create(&redemption).Error) | |||
| quota, err := Redeem("local-redeem-key", 101) | |||
| require.NoError(t, err) | |||
| assert.Equal(t, 50000, quota) | |||
| var updatedUser User | |||
| require.NoError(t, db.First(&updatedUser, 101).Error) | |||
| assert.Equal(t, 150000, updatedUser.Quota) | |||
| } | |||
| @@ -0,0 +1,314 @@ | |||
| package model | |||
| import ( | |||
| "errors" | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| func setupRegionMigrationUserDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, err := db.DB() | |||
| require.NoError(t, err) | |||
| sqlDB.SetMaxOpenConns(1) | |||
| require.NoError(t, db.AutoMigrate(&User{}, &CustomOAuthProvider{}, &UserOAuthBinding{})) | |||
| return db | |||
| } | |||
| func TestInsertImportedUserWithHashedPassword_PreservesHash(t *testing.T) { | |||
| db := setupRegionMigrationUserDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| user := &User{ | |||
| Username: "ov-alice", | |||
| Password: "already-hashed-value", | |||
| Email: "alice@example.com", | |||
| DisplayName: "Alice", | |||
| Status: common.UserStatusEnabled, | |||
| Role: common.RoleCommonUser, | |||
| Group: "default", | |||
| Quota: 123, | |||
| AffCode: "A001", | |||
| CreatedAt: 1710000000, | |||
| } | |||
| require.NoError(t, InsertImportedUserWithHashedPassword(user)) | |||
| var saved User | |||
| require.NoError(t, db.Where("username = ?", "ov-alice").First(&saved).Error) | |||
| require.Equal(t, "already-hashed-value", saved.Password) | |||
| require.Greater(t, saved.Id, 0) | |||
| } | |||
| func TestInsertImportedUserWithHashedPassword_RejectsExplicitId(t *testing.T) { | |||
| db := setupRegionMigrationUserDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| user := &User{ | |||
| Id: 10000001, | |||
| Username: "bad-import", | |||
| Password: "hash", | |||
| AffCode: "X001", | |||
| } | |||
| err := InsertImportedUserWithHashedPassword(user) | |||
| require.Error(t, err) | |||
| require.Contains(t, err.Error(), "Id must not be set") | |||
| } | |||
| func TestInsertImportedUserWithHashedPassword_RejectsOverlongFields(t *testing.T) { | |||
| db := setupRegionMigrationUserDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| user := &User{ | |||
| Username: "abcdefghijklmnopqrstu", | |||
| Password: "already-hashed-value", | |||
| Email: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx@example.com", | |||
| DisplayName: "DDDDDDDDDDDDDDDDDDDDDDDDD", | |||
| Status: common.UserStatusEnabled, | |||
| Role: common.RoleCommonUser, | |||
| Group: "default", | |||
| Quota: 123, | |||
| AffCode: "A002", | |||
| CreatedAt: 1710000000, | |||
| } | |||
| err := InsertImportedUserWithHashedPassword(user) | |||
| require.Error(t, err) | |||
| require.Contains(t, err.Error(), "username exceeds max length") | |||
| user.Username = "valid-user" | |||
| err = InsertImportedUserWithHashedPassword(user) | |||
| require.Error(t, err) | |||
| require.Contains(t, err.Error(), "display_name exceeds max length") | |||
| user.DisplayName = "Valid Name" | |||
| err = InsertImportedUserWithHashedPassword(user) | |||
| require.Error(t, err) | |||
| require.Contains(t, err.Error(), "email exceeds max length") | |||
| } | |||
| func TestCopyOAuthBindingsIfMissing_Idempotent(t *testing.T) { | |||
| db := setupRegionMigrationUserDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&CustomOAuthProvider{ | |||
| Id: 10, | |||
| Name: "GitHub Enterprise", | |||
| Slug: "gh-enterprise", | |||
| Enabled: true, | |||
| ClientId: "x", | |||
| AuthorizationEndpoint: "a", | |||
| TokenEndpoint: "t", | |||
| UserInfoEndpoint: "u", | |||
| }).Error) | |||
| require.NoError(t, db.Create(&User{Id: 1, Username: "cn-user", Password: "hash", AffCode: "AFF1"}).Error) | |||
| bindings := []*UserOAuthBinding{{ | |||
| ProviderId: 10, | |||
| ProviderUserId: "u-100", | |||
| }} | |||
| require.NoError(t, CopyOAuthBindingsIfMissing(1, bindings)) | |||
| require.NoError(t, CopyOAuthBindingsIfMissing(1, bindings)) | |||
| var count int64 | |||
| require.NoError(t, db.Model(&UserOAuthBinding{}).Where("user_id = ?", 1).Count(&count).Error) | |||
| require.EqualValues(t, 1, count) | |||
| } | |||
| func TestConvertUserToSynced_Idempotent(t *testing.T) { | |||
| db := setupRegionMigrationUserDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&User{ | |||
| Id: 10000001, | |||
| Username: "ov-user", | |||
| Password: "hash", | |||
| AffCode: "AFF2", | |||
| Quota: 88, | |||
| Source: common.UserSourceLocal, | |||
| }).Error) | |||
| require.NoError(t, ConvertUserToSynced(10000001, 2001, 88)) | |||
| require.NoError(t, ConvertUserToSynced(10000001, 2001, 88)) | |||
| var saved User | |||
| require.NoError(t, db.First(&saved, 10000001).Error) | |||
| require.Equal(t, common.UserSourceSynced, saved.Source) | |||
| require.Equal(t, 2001, saved.RemoteUserId) | |||
| } | |||
| func TestConvertUserToSynced_DifferentRemoteUserIdFails(t *testing.T) { | |||
| db := setupRegionMigrationUserDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&User{ | |||
| Id: 10000002, | |||
| Username: "ov-user2", | |||
| Password: "hash", | |||
| AffCode: "AFF3", | |||
| Source: common.UserSourceLocal, | |||
| }).Error) | |||
| require.NoError(t, ConvertUserToSynced(10000002, 3001, 100)) | |||
| err := ConvertUserToSynced(10000002, 9999, 100) | |||
| require.Error(t, err) | |||
| require.ErrorIs(t, err, ErrSyncedUserRemoteUserIDImmutable) | |||
| } | |||
| func TestListLocalUsersForMigration_FiltersLocalOnly(t *testing.T) { | |||
| db := setupRegionMigrationUserDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&User{ | |||
| Id: 10000001, | |||
| Username: "local-1", | |||
| Password: "h", | |||
| AffCode: "L1", | |||
| Source: common.UserSourceLocal, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&User{ | |||
| Id: 10000002, | |||
| Username: "synced-1", | |||
| Password: "h", | |||
| AffCode: "S1", | |||
| Source: common.UserSourceSynced, | |||
| RemoteUserId: 1, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&User{ | |||
| Id: 10000003, | |||
| Username: "root", | |||
| Password: "h", | |||
| AffCode: "R1", | |||
| Source: common.UserSourceLocal, | |||
| Role: common.RoleRootUser, | |||
| }).Error) | |||
| users, total, err := ListLocalUsersForMigration(0, 20) | |||
| require.NoError(t, err) | |||
| require.EqualValues(t, 1, total) | |||
| require.Len(t, users, 1) | |||
| require.Equal(t, "local-1", users[0].Username) | |||
| } | |||
| func TestQueryLocalUsersForMigration_FiltersByKeyword(t *testing.T) { | |||
| db := setupRegionMigrationUserDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&User{ | |||
| Id: 10000011, | |||
| Username: "alice-local", | |||
| Password: "h", | |||
| Email: "alice@example.com", | |||
| AffCode: "AL1", | |||
| Source: common.UserSourceLocal, | |||
| }).Error) | |||
| require.NoError(t, db.Create(&User{ | |||
| Id: 10000012, | |||
| Username: "bob-local", | |||
| Password: "h", | |||
| Email: "bob@example.com", | |||
| AffCode: "BO1", | |||
| Source: common.UserSourceLocal, | |||
| }).Error) | |||
| users, total, err := QueryLocalUsersForMigration(0, 20, "alice") | |||
| require.NoError(t, err) | |||
| require.EqualValues(t, 1, total) | |||
| require.Len(t, users, 1) | |||
| require.Equal(t, "alice-local", users[0].Username) | |||
| } | |||
| func TestConvertUserToSynced_RejectsRootUser(t *testing.T) { | |||
| db := setupRegionMigrationUserDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&User{ | |||
| Id: 10000010, | |||
| Username: "root", | |||
| Password: "hash", | |||
| AffCode: "ROOT1", | |||
| Source: common.UserSourceLocal, | |||
| Role: common.RoleRootUser, | |||
| }).Error) | |||
| err := ConvertUserToSynced(10000010, 2001, 88) | |||
| require.Error(t, err) | |||
| require.ErrorIs(t, err, ErrConvertRootUserToSynced) | |||
| var saved User | |||
| require.NoError(t, db.First(&saved, 10000010).Error) | |||
| require.Equal(t, common.UserSourceLocal, saved.Source) | |||
| require.Equal(t, 0, saved.RemoteUserId) | |||
| } | |||
| func TestConvertUserToSynced_SentinelErrorsRemainDistinct(t *testing.T) { | |||
| db := setupRegionMigrationUserDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&User{ | |||
| Id: 10000020, | |||
| Username: "already-synced", | |||
| Password: "hash", | |||
| AffCode: "SYNC20", | |||
| Source: common.UserSourceSynced, | |||
| RemoteUserId: 3001, | |||
| }).Error) | |||
| err := ConvertUserToSynced(10000020, 9999, 100) | |||
| require.Error(t, err) | |||
| require.True(t, errors.Is(err, ErrSyncedUserRemoteUserIDImmutable)) | |||
| require.False(t, errors.Is(err, ErrConvertRootUserToSynced)) | |||
| } | |||
| @@ -0,0 +1,45 @@ | |||
| package model | |||
| import ( | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| type channelPricingMigrationStub struct { | |||
| ID uint `gorm:"primaryKey"` | |||
| } | |||
| func (channelPricingMigrationStub) TableName() string { | |||
| return "channel_pricings" | |||
| } | |||
| type pricingTagMigrationStub struct { | |||
| ID uint `gorm:"primaryKey"` | |||
| } | |||
| func (pricingTagMigrationStub) TableName() string { | |||
| return "pricing_tags" | |||
| } | |||
| func TestMigrateDBDropsChannelPricingTables(t *testing.T) { | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| origDB := DB | |||
| origSQLite := common.UsingSQLite | |||
| DB = db | |||
| common.UsingSQLite = true | |||
| t.Cleanup(func() { | |||
| DB = origDB | |||
| common.UsingSQLite = origSQLite | |||
| }) | |||
| require.NoError(t, db.AutoMigrate(&channelPricingMigrationStub{}, &pricingTagMigrationStub{})) | |||
| require.NoError(t, migrateDB()) | |||
| require.False(t, DB.Migrator().HasTable("channel_pricings")) | |||
| require.False(t, DB.Migrator().HasTable("pricing_tags")) | |||
| } | |||
| @@ -7,6 +7,7 @@ import ( | |||
| "strconv" | |||
| "strings" | |||
| "time" | |||
| "unicode/utf8" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/dto" | |||
| @@ -55,6 +56,7 @@ type User struct { | |||
| AffQuota int `json:"aff_quota" gorm:"type:int;default:0;column:aff_quota"` // 邀请剩余额度 | |||
| AffHistoryQuota int `json:"aff_history_quota" gorm:"type:int;default:0;column:aff_history"` // 邀请历史额度 | |||
| InviterId int `json:"inviter_id" gorm:"type:int;column:inviter_id;index"` | |||
| CreatedAt int64 `json:"created_at" gorm:"bigint;default:0"` | |||
| DeletedAt gorm.DeletedAt `gorm:"index"` | |||
| LinuxDOId string `json:"linux_do_id" gorm:"column:linux_do_id;index"` | |||
| Setting string `json:"setting" gorm:"type:text;column:setting"` | |||
| @@ -347,13 +349,66 @@ func GetUserById(id int, selectAll bool) (*User, error) { | |||
| return &user, err | |||
| } | |||
| func getUserByStringColumn(column string, value string) (*User, error) { | |||
| if value == "" { | |||
| return nil, nil | |||
| } | |||
| var user User | |||
| result := DB.Where(column+" = ?", value).Limit(1).Find(&user) | |||
| if result.Error != nil { | |||
| return nil, result.Error | |||
| } | |||
| if result.RowsAffected == 0 { | |||
| return nil, nil | |||
| } | |||
| return &user, nil | |||
| } | |||
| func GetUserByUsername(username string) (*User, error) { | |||
| return getUserByStringColumn("username", username) | |||
| } | |||
| func GetUserByEmail(email string) (*User, error) { | |||
| return getUserByStringColumn("email", email) | |||
| } | |||
| func GetUserByGitHubId(githubId string) (*User, error) { | |||
| return getUserByStringColumn("github_id", githubId) | |||
| } | |||
| func GetUserByDiscordId(discordId string) (*User, error) { | |||
| return getUserByStringColumn("discord_id", discordId) | |||
| } | |||
| func GetUserByOidcId(oidcId string) (*User, error) { | |||
| return getUserByStringColumn("oidc_id", oidcId) | |||
| } | |||
| func GetUserByWeChatId(wechatId string) (*User, error) { | |||
| return getUserByStringColumn("wechat_id", wechatId) | |||
| } | |||
| func GetUserByTelegramId(telegramId string) (*User, error) { | |||
| return getUserByStringColumn("telegram_id", telegramId) | |||
| } | |||
| func GetUserByLinuxDOId(linuxDOId string) (*User, error) { | |||
| return getUserByStringColumn("linux_do_id", linuxDOId) | |||
| } | |||
| func GetUserIdByAffCode(affCode string) (int, error) { | |||
| if affCode == "" { | |||
| return 0, errors.New("affCode 为空!") | |||
| } | |||
| var user User | |||
| err := DB.Select("id").First(&user, "aff_code = ?", affCode).Error | |||
| return user.Id, err | |||
| result := DB.Select("id").Where("aff_code = ?", affCode).Limit(1).Find(&user) | |||
| if result.Error != nil { | |||
| return 0, result.Error | |||
| } | |||
| if result.RowsAffected == 0 { | |||
| return 0, gorm.ErrRecordNotFound | |||
| } | |||
| return user.Id, nil | |||
| } | |||
| func DeleteUserById(id int) (err error) { | |||
| @@ -436,6 +491,7 @@ func (user *User) Insert(inviterId int) error { | |||
| } | |||
| //user.SetAccessToken(common.GetUUID()) | |||
| user.AffCode = common.GetRandomString(4) | |||
| user.CreatedAt = time.Now().Unix() | |||
| // 初始化用户设置,包括默认的边栏配置 | |||
| if user.Setting == "" { | |||
| @@ -503,6 +559,7 @@ func (user *User) InsertWithTx(tx *gorm.DB, inviterId int) error { | |||
| user.Quota = common.QuotaForNewUser | |||
| } | |||
| user.AffCode = common.GetRandomString(4) | |||
| user.CreatedAt = time.Now().Unix() | |||
| // 初始化用户设置 | |||
| if user.Setting == "" { | |||
| @@ -518,6 +575,53 @@ func (user *User) InsertWithTx(tx *gorm.DB, inviterId int) error { | |||
| return nil | |||
| } | |||
| // InsertImportedUserWithHashedPassword 专用于迁移导入,直接写入已有密码哈希。 | |||
| func InsertImportedUserWithHashedPassword(user *User) error { | |||
| return InsertImportedUserWithHashedPasswordTx(DB, user) | |||
| } | |||
| func validateImportedUserFields(user *User) error { | |||
| if user == nil { | |||
| return errors.New("user is nil") | |||
| } | |||
| if strings.TrimSpace(user.Username) == "" { | |||
| return errors.New("username is required") | |||
| } | |||
| if utf8.RuneCountInString(user.Username) > UserNameMaxLength { | |||
| return fmt.Errorf("username exceeds max length %d", UserNameMaxLength) | |||
| } | |||
| if utf8.RuneCountInString(user.DisplayName) > 20 { | |||
| return errors.New("display_name exceeds max length 20") | |||
| } | |||
| if utf8.RuneCountInString(user.Email) > 50 { | |||
| return errors.New("email exceeds max length 50") | |||
| } | |||
| return nil | |||
| } | |||
| func InsertImportedUserWithHashedPasswordTx(tx *gorm.DB, user *User) error { | |||
| if user == nil { | |||
| return errors.New("user is nil") | |||
| } | |||
| if user.Id != 0 { | |||
| return errors.New("InsertImportedUserWithHashedPassword: Id must not be set, let DB auto-assign") | |||
| } | |||
| if err := validateImportedUserFields(user); err != nil { | |||
| return err | |||
| } | |||
| if user.AffCode == "" { | |||
| user.AffCode = common.GetRandomString(4) | |||
| } | |||
| if user.CreatedAt == 0 { | |||
| user.CreatedAt = common.GetTimestamp() | |||
| } | |||
| if user.Setting == "" { | |||
| defaultSetting := dto.UserSetting{} | |||
| user.SetSetting(defaultSetting) | |||
| } | |||
| return tx.Create(user).Error | |||
| } | |||
| // FinalizeOAuthUserCreation performs post-transaction tasks for OAuth user creation. | |||
| // This should be called after the transaction commits successfully. | |||
| func (user *User) FinalizeOAuthUserCreation(inviterId int) { | |||
| @@ -1017,6 +1121,85 @@ func DeltaUpdateUserQuota(id int, delta int) (err error) { | |||
| } | |||
| } | |||
| func ListLocalUsersForMigration(page, pageSize int) ([]*User, int64, error) { | |||
| return QueryLocalUsersForMigration(page, pageSize, "") | |||
| } | |||
| func QueryLocalUsersForMigration(page, pageSize int, keyword string) ([]*User, int64, error) { | |||
| if page < 0 { | |||
| page = 0 | |||
| } | |||
| if pageSize <= 0 { | |||
| pageSize = 20 | |||
| } | |||
| var users []*User | |||
| var total int64 | |||
| query := DB.Model(&User{}). | |||
| Where("source = ?", common.UserSourceLocal). | |||
| Where("role <> ?", common.RoleRootUser) | |||
| keyword = strings.TrimSpace(keyword) | |||
| if keyword != "" { | |||
| likeKeyword := "%" + strings.ToLower(keyword) + "%" | |||
| if id, err := strconv.Atoi(keyword); err == nil { | |||
| query = query.Where("id = ? OR lower(username) LIKE ? OR lower(email) LIKE ?", id, likeKeyword, likeKeyword) | |||
| } else { | |||
| query = query.Where("lower(username) LIKE ? OR lower(email) LIKE ?", likeKeyword, likeKeyword) | |||
| } | |||
| } | |||
| if err := query.Count(&total).Error; err != nil { | |||
| return nil, 0, err | |||
| } | |||
| err := query.Order("id asc").Offset(page * pageSize).Limit(pageSize).Find(&users).Error | |||
| return users, total, err | |||
| } | |||
| func HasOVSyncedCopyByRemoteId(cnUserId int) (bool, error) { | |||
| var count int64 | |||
| err := DB.Model(&User{}). | |||
| Where("remote_user_id = ? AND source = ?", cnUserId, common.UserSourceSynced). | |||
| Count(&count).Error | |||
| return count > 0, err | |||
| } | |||
| var ( | |||
| ErrConvertRootUserToSynced = errors.New("user.convert_root_to_synced_denied") | |||
| ErrSyncedUserRemoteUserIDImmutable = errors.New("user.synced_remote_user_id_immutable") | |||
| ) | |||
| func ConvertUserToSynced(userId, remoteUserId, syncedQuota int) error { | |||
| var user User | |||
| if err := DB.Where("id = ?", userId).First(&user).Error; err != nil { | |||
| return err | |||
| } | |||
| if user.Role == common.RoleRootUser { | |||
| return ErrConvertRootUserToSynced | |||
| } | |||
| now := common.GetTimestamp() | |||
| if user.Source == common.UserSourceSynced { | |||
| if user.RemoteUserId != remoteUserId { | |||
| return fmt.Errorf("%w: user %d is already synced to remote_user_id=%d, cannot change to %d", | |||
| ErrSyncedUserRemoteUserIDImmutable, | |||
| userId, user.RemoteUserId, remoteUserId) | |||
| } | |||
| if user.SyncedQuota == syncedQuota { | |||
| return nil | |||
| } | |||
| return DB.Model(&User{}).Where("id = ?", userId).Updates(map[string]any{ | |||
| "synced_quota": syncedQuota, | |||
| "last_sync_at": now, | |||
| }).Error | |||
| } | |||
| return DB.Model(&User{}).Where("id = ?", userId).Updates(map[string]any{ | |||
| "source": common.UserSourceSynced, | |||
| "remote_user_id": remoteUserId, | |||
| "synced_quota": syncedQuota, | |||
| "last_sync_at": now, | |||
| }).Error | |||
| } | |||
| // GetSyncedUsers 获取所有同步用户 | |||
| func GetSyncedUsers() []User { | |||
| var users []User | |||
| @@ -1,129 +0,0 @@ | |||
| package model | |||
| import ( | |||
| "strconv" | |||
| "strings" | |||
| "sync" | |||
| "github.com/QuantumNous/new-api/common" | |||
| ) | |||
| var ( | |||
| userChannelRatioCache = make(map[string]float64) // key: "userId:modelName:channelId" -> ratio | |||
| userChannelRatioCacheLock sync.RWMutex | |||
| ) | |||
| // UserChannelRatio 用户-模型-渠道倍率表 | |||
| type UserChannelRatio struct { | |||
| Id int `json:"id" gorm:"primaryKey"` | |||
| UserId int `json:"user_id" gorm:"not null;uniqueIndex:idx_user_model_channel,priority:1"` | |||
| ModelName string `json:"model_name" gorm:"size:128;not null;uniqueIndex:idx_user_model_channel,priority:2"` | |||
| ChannelId int `json:"channel_id" gorm:"not null;uniqueIndex:idx_user_model_channel,priority:3"` | |||
| Ratio float64 `json:"ratio" gorm:"default:1"` | |||
| CreatedAt int64 `json:"created_at" gorm:"bigint"` | |||
| UpdatedAt int64 `json:"updated_at" gorm:"bigint"` | |||
| } | |||
| func getUserChannelRatioCacheKey(userId int, modelName string, channelId int) string { | |||
| return strconv.Itoa(userId) + ":" + modelName + ":" + strconv.Itoa(channelId) | |||
| } | |||
| func setUserChannelRatioCache(key string, ratio float64) { | |||
| userChannelRatioCacheLock.Lock() | |||
| userChannelRatioCache[key] = ratio | |||
| userChannelRatioCacheLock.Unlock() | |||
| } | |||
| func removeUserChannelRatioCache(key string) { | |||
| userChannelRatioCacheLock.Lock() | |||
| delete(userChannelRatioCache, key) | |||
| userChannelRatioCacheLock.Unlock() | |||
| } | |||
| // GetUserChannelRatio 获取用户在指定模型+渠道的倍率(纯内存读) | |||
| // 未命中返回 1.0(不影响计费) | |||
| func GetUserChannelRatio(userId int, modelName string, channelId int) float64 { | |||
| key := getUserChannelRatioCacheKey(userId, modelName, channelId) | |||
| userChannelRatioCacheLock.RLock() | |||
| ratio, ok := userChannelRatioCache[key] | |||
| userChannelRatioCacheLock.RUnlock() | |||
| if !ok { | |||
| return 1.0 | |||
| } | |||
| return ratio | |||
| } | |||
| // LoadUserChannelRatioCache 全量加载到内存(启动时调用) | |||
| func LoadUserChannelRatioCache() { | |||
| var records []*UserChannelRatio | |||
| if err := DB.Find(&records).Error; err != nil { | |||
| common.SysError("[UserChannelRatio] LoadCache failed: " + err.Error()) | |||
| return | |||
| } | |||
| userChannelRatioCacheLock.Lock() | |||
| userChannelRatioCache = make(map[string]float64, len(records)) | |||
| for _, r := range records { | |||
| key := getUserChannelRatioCacheKey(r.UserId, r.ModelName, r.ChannelId) | |||
| userChannelRatioCache[key] = r.Ratio | |||
| } | |||
| userChannelRatioCacheLock.Unlock() | |||
| common.SysLog("[UserChannelRatio] cache loaded " + strconv.Itoa(len(records)) + " records") | |||
| } | |||
| func (ucr *UserChannelRatio) Insert() error { | |||
| ucr.CreatedAt = common.GetTimestamp() | |||
| ucr.UpdatedAt = common.GetTimestamp() | |||
| err := DB.Create(ucr).Error | |||
| if err == nil { | |||
| setUserChannelRatioCache(getUserChannelRatioCacheKey(ucr.UserId, ucr.ModelName, ucr.ChannelId), ucr.Ratio) | |||
| } | |||
| return err | |||
| } | |||
| func (ucr *UserChannelRatio) Update() error { | |||
| var existing UserChannelRatio | |||
| if err := DB.First(&existing, ucr.Id).Error; err != nil { | |||
| return err | |||
| } | |||
| ucr.UpdatedAt = common.GetTimestamp() | |||
| err := DB.Model(&UserChannelRatio{}).Where("id = ?", ucr.Id). | |||
| Select("ratio", "updated_at"). | |||
| Updates(ucr).Error | |||
| if err == nil { | |||
| setUserChannelRatioCache(getUserChannelRatioCacheKey(existing.UserId, existing.ModelName, existing.ChannelId), ucr.Ratio) | |||
| } | |||
| return err | |||
| } | |||
| func DeleteUserChannelRatioById(id int) error { | |||
| var existing UserChannelRatio | |||
| if err := DB.First(&existing, id).Error; err != nil { | |||
| return err | |||
| } | |||
| err := DB.Delete(&existing).Error | |||
| if err == nil { | |||
| removeUserChannelRatioCache(getUserChannelRatioCacheKey(existing.UserId, existing.ModelName, existing.ChannelId)) | |||
| } | |||
| return err | |||
| } | |||
| // GetUserChannelRatiosByUserId 获取指定用户的所有倍率记录 | |||
| func GetUserChannelRatiosByUserId(userId int) ([]*UserChannelRatio, error) { | |||
| var list []*UserChannelRatio | |||
| err := DB.Where("user_id = ?", userId).Find(&list).Error | |||
| return list, err | |||
| } | |||
| // GetBestUserChannelRatio 从内存缓存中查找指定用户+模型的最低渠道倍率(未命中返回 1.0) | |||
| func GetBestUserChannelRatio(userId int, modelName string) float64 { | |||
| best := 1.0 | |||
| prefix := strconv.Itoa(userId) + ":" + modelName + ":" | |||
| userChannelRatioCacheLock.RLock() | |||
| for key, ratio := range userChannelRatioCache { | |||
| if len(key) > len(prefix) && strings.HasPrefix(key, prefix) && ratio < best { | |||
| best = ratio | |||
| } | |||
| } | |||
| userChannelRatioCacheLock.RUnlock() | |||
| return best | |||
| } | |||
| @@ -1,105 +0,0 @@ | |||
| package model | |||
| import ( | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/assert" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| func setupUserChannelRatioDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, _ := db.DB() | |||
| sqlDB.SetMaxOpenConns(1) | |||
| origDB := DB | |||
| DB = db | |||
| common.UsingSQLite = true | |||
| common.RedisEnabled = false | |||
| require.NoError(t, db.AutoMigrate(&UserChannelRatio{})) | |||
| t.Cleanup(func() { | |||
| DB = origDB | |||
| sqlDB.Close() | |||
| }) | |||
| return db | |||
| } | |||
| func TestUserChannelRatioCRUD(t *testing.T) { | |||
| setupUserChannelRatioDB(t) | |||
| LoadUserChannelRatioCache() | |||
| ucr := &UserChannelRatio{ | |||
| UserId: 1, | |||
| ModelName: "gpt-4o", | |||
| ChannelId: 5, | |||
| Ratio: 0.8, | |||
| } | |||
| err := ucr.Insert() | |||
| require.NoError(t, err) | |||
| assert.True(t, ucr.Id > 0) | |||
| ratio := GetUserChannelRatio(1, "gpt-4o", 5) | |||
| assert.Equal(t, 0.8, ratio) | |||
| ratio = GetUserChannelRatio(999, "nonexistent", 999) | |||
| assert.Equal(t, 1.0, ratio) | |||
| ucr.Ratio = 1.2 | |||
| err = ucr.Update() | |||
| require.NoError(t, err) | |||
| ratio = GetUserChannelRatio(1, "gpt-4o", 5) | |||
| assert.Equal(t, 1.2, ratio) | |||
| list, err := GetUserChannelRatiosByUserId(1) | |||
| require.NoError(t, err) | |||
| assert.Len(t, list, 1) | |||
| assert.Equal(t, "gpt-4o", list[0].ModelName) | |||
| err = DeleteUserChannelRatioById(ucr.Id) | |||
| require.NoError(t, err) | |||
| ratio = GetUserChannelRatio(1, "gpt-4o", 5) | |||
| assert.Equal(t, 1.0, ratio) | |||
| list, err = GetUserChannelRatiosByUserId(1) | |||
| require.NoError(t, err) | |||
| assert.Len(t, list, 0) | |||
| } | |||
| func TestUserChannelRatioUniqueConstraint(t *testing.T) { | |||
| setupUserChannelRatioDB(t) | |||
| LoadUserChannelRatioCache() | |||
| ucr1 := &UserChannelRatio{UserId: 1, ModelName: "gpt-4o", ChannelId: 5, Ratio: 0.8} | |||
| err := ucr1.Insert() | |||
| require.NoError(t, err) | |||
| ucr2 := &UserChannelRatio{UserId: 1, ModelName: "gpt-4o", ChannelId: 5, Ratio: 1.0} | |||
| err = ucr2.Insert() | |||
| assert.Error(t, err) | |||
| ucr3 := &UserChannelRatio{UserId: 1, ModelName: "gpt-4o", ChannelId: 6, Ratio: 1.5} | |||
| err = ucr3.Insert() | |||
| assert.NoError(t, err) | |||
| } | |||
| func TestLoadUserChannelRatioCache(t *testing.T) { | |||
| setupUserChannelRatioDB(t) | |||
| ucr1 := &UserChannelRatio{UserId: 1, ModelName: "gpt-4o", ChannelId: 5, Ratio: 0.8} | |||
| ucr2 := &UserChannelRatio{UserId: 2, ModelName: "claude-3", ChannelId: 10, Ratio: 1.5} | |||
| require.NoError(t, ucr1.Insert()) | |||
| require.NoError(t, ucr2.Insert()) | |||
| LoadUserChannelRatioCache() | |||
| assert.Equal(t, 0.8, GetUserChannelRatio(1, "gpt-4o", 5)) | |||
| assert.Equal(t, 1.5, GetUserChannelRatio(2, "claude-3", 10)) | |||
| assert.Equal(t, 1.0, GetUserChannelRatio(3, "nonexistent", 99)) | |||
| } | |||
| @@ -0,0 +1,377 @@ | |||
| package model | |||
| import ( | |||
| "fmt" | |||
| "slices" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "gorm.io/gorm" | |||
| "gorm.io/gorm/clause" | |||
| ) | |||
| const ( | |||
| UserMigrationBatchStatusDraft = "draft" | |||
| UserMigrationBatchStatusScanning = "scanning" | |||
| UserMigrationBatchStatusScanned = "scanned" | |||
| UserMigrationBatchStatusReady = "ready" | |||
| UserMigrationBatchStatusRunning = "running" | |||
| UserMigrationBatchStatusCompleted = "completed" | |||
| UserMigrationBatchStatusFailed = "failed" | |||
| UserMigrationBatchStatusCancelled = "cancelled" | |||
| UserMigrationItemStatusPendingScan = "pending_scan" | |||
| UserMigrationItemStatusConflict = "conflict" | |||
| UserMigrationItemStatusReady = "ready" | |||
| UserMigrationItemStatusMigrating = "migrating" | |||
| UserMigrationItemStatusMigrated = "migrated" | |||
| UserMigrationItemStatusSkipped = "skipped" | |||
| UserMigrationItemStatusFailed = "failed" | |||
| UserMigrationStrategyCreateNew = "create_new" | |||
| UserMigrationStrategyMergeExisting = "merge_into_existing" | |||
| UserMigrationStrategySkip = "skip" | |||
| ) | |||
| type UserMigrationBatch struct { | |||
| Id int `json:"id" gorm:"primaryKey"` | |||
| Name string `json:"name" gorm:"type:varchar(128);not null"` | |||
| SourceRegion string `json:"source_region" gorm:"type:varchar(32);not null"` | |||
| TargetRegion string `json:"target_region" gorm:"type:varchar(32);not null"` | |||
| SelectionMode string `json:"selection_mode" gorm:"type:varchar(32);not null;default:'all'"` | |||
| SelectionPayload string `json:"selection_payload" gorm:"type:text"` | |||
| RequestedUserCount int `json:"requested_user_count" gorm:"default:0"` | |||
| MatchedUserCount int `json:"matched_user_count" gorm:"default:0"` | |||
| ExcludedUserCount int `json:"excluded_user_count" gorm:"default:0"` | |||
| SelectionSummary string `json:"selection_summary" gorm:"type:text"` | |||
| Status string `json:"status" gorm:"type:varchar(32);not null;default:'draft';index"` | |||
| TotalUsers int `json:"total_users" gorm:"default:0"` | |||
| ConflictUsers int `json:"conflict_users" gorm:"default:0"` | |||
| MigratedUsers int `json:"migrated_users" gorm:"default:0"` | |||
| FailedUsers int `json:"failed_users" gorm:"default:0"` | |||
| OperatorId int `json:"operator_id" gorm:"index"` | |||
| CreatedAt int64 `json:"created_at" gorm:"bigint"` | |||
| UpdatedAt int64 `json:"updated_at" gorm:"bigint"` | |||
| } | |||
| type UserMigrationItem struct { | |||
| Id int `json:"id" gorm:"primaryKey"` | |||
| BatchId int `json:"batch_id" gorm:"index:idx_batch_source_user,unique"` | |||
| SourceUserId int `json:"source_user_id" gorm:"index:idx_batch_source_user,unique"` | |||
| TargetUserId int `json:"target_user_id" gorm:"default:0;index"` | |||
| SourceUsername string `json:"source_username" gorm:"type:varchar(128);not null"` | |||
| SourceEmail string `json:"source_email" gorm:"type:varchar(255);default:''"` | |||
| SourceQuota int `json:"source_quota" gorm:"default:0"` | |||
| ConflictFlags string `json:"conflict_flags" gorm:"type:text"` | |||
| ResolutionStrategy string `json:"resolution_strategy" gorm:"type:varchar(32);default:''"` | |||
| Status string `json:"status" gorm:"type:varchar(32);not null;default:'pending_scan';index"` | |||
| ErrorMessage string `json:"error_message" gorm:"type:text"` | |||
| Snapshot string `json:"snapshot" gorm:"type:text"` | |||
| MigrationToken string `json:"migration_token" gorm:"type:varchar(64);default:'';index"` | |||
| MigratedAt int64 `json:"migrated_at" gorm:"bigint;default:0"` | |||
| CreatedAt int64 `json:"created_at" gorm:"bigint"` | |||
| UpdatedAt int64 `json:"updated_at" gorm:"bigint"` | |||
| } | |||
| type MigrationQuotaGrant struct { | |||
| Id int `json:"id" gorm:"primaryKey"` | |||
| MigrationToken string `json:"migration_token" gorm:"type:varchar(64);not null;uniqueIndex"` | |||
| CreatedAt int64 `json:"created_at" gorm:"bigint"` | |||
| } | |||
| type ActiveMigrationUserConflict struct { | |||
| SourceUserID int `json:"source_user_id"` | |||
| BatchID int `json:"batch_id"` | |||
| BatchStatus string `json:"batch_status"` | |||
| } | |||
| func (b *UserMigrationBatch) BeforeCreate(tx *gorm.DB) error { | |||
| now := common.GetTimestamp() | |||
| if b.Status == "" { | |||
| b.Status = UserMigrationBatchStatusDraft | |||
| } | |||
| if b.SelectionMode == "" { | |||
| b.SelectionMode = "all" | |||
| } | |||
| b.CreatedAt = now | |||
| b.UpdatedAt = now | |||
| return nil | |||
| } | |||
| func (b *UserMigrationBatch) BeforeUpdate(tx *gorm.DB) error { | |||
| b.UpdatedAt = common.GetTimestamp() | |||
| return nil | |||
| } | |||
| func (i *UserMigrationItem) BeforeCreate(tx *gorm.DB) error { | |||
| now := common.GetTimestamp() | |||
| if i.Status == "" { | |||
| i.Status = UserMigrationItemStatusPendingScan | |||
| } | |||
| i.CreatedAt = now | |||
| i.UpdatedAt = now | |||
| return nil | |||
| } | |||
| func (i *UserMigrationItem) BeforeUpdate(tx *gorm.DB) error { | |||
| i.UpdatedAt = common.GetTimestamp() | |||
| return nil | |||
| } | |||
| func (g *MigrationQuotaGrant) BeforeCreate(tx *gorm.DB) error { | |||
| g.CreatedAt = common.GetTimestamp() | |||
| return nil | |||
| } | |||
| func CreateUserMigrationBatch(batch *UserMigrationBatch) error { | |||
| return DB.Create(batch).Error | |||
| } | |||
| func CancelUserMigrationBatch(batchId int) (bool, error) { | |||
| var batch UserMigrationBatch | |||
| if err := DB.First(&batch, batchId).Error; err != nil { | |||
| return false, err | |||
| } | |||
| switch batch.Status { | |||
| case UserMigrationBatchStatusCancelled: | |||
| return true, nil | |||
| case UserMigrationBatchStatusDraft, | |||
| UserMigrationBatchStatusScanned, | |||
| UserMigrationBatchStatusReady, | |||
| UserMigrationBatchStatusFailed: | |||
| return true, DB.Model(&UserMigrationBatch{}). | |||
| Where("id = ?", batchId). | |||
| Update("status", UserMigrationBatchStatusCancelled).Error | |||
| default: | |||
| return false, nil | |||
| } | |||
| } | |||
| func FindActiveMigrationUserConflicts(sourceUserIDs []int) ([]ActiveMigrationUserConflict, error) { | |||
| if len(sourceUserIDs) == 0 { | |||
| return nil, nil | |||
| } | |||
| activeStatuses := []string{ | |||
| UserMigrationBatchStatusDraft, | |||
| UserMigrationBatchStatusScanning, | |||
| UserMigrationBatchStatusScanned, | |||
| UserMigrationBatchStatusReady, | |||
| UserMigrationBatchStatusRunning, | |||
| UserMigrationBatchStatusFailed, | |||
| } | |||
| type activeBatchSelection struct { | |||
| ID int | |||
| Status string | |||
| SelectionMode string | |||
| SelectionPayload string | |||
| } | |||
| conflictMap := make(map[string]ActiveMigrationUserConflict) | |||
| addConflict := func(conflict ActiveMigrationUserConflict) { | |||
| key := fmt.Sprintf("%d:%d", conflict.SourceUserID, conflict.BatchID) | |||
| if _, exists := conflictMap[key]; exists { | |||
| return | |||
| } | |||
| conflictMap[key] = conflict | |||
| } | |||
| var itemConflicts []ActiveMigrationUserConflict | |||
| err := DB.Table("user_migration_items AS umi"). | |||
| Select("umi.source_user_id, umb.id AS batch_id, umb.status AS batch_status"). | |||
| Joins("JOIN user_migration_batches AS umb ON umb.id = umi.batch_id"). | |||
| Where("umi.source_user_id IN ?", sourceUserIDs). | |||
| Where("umb.status IN ?", activeStatuses). | |||
| Order("umi.source_user_id ASC, umb.id ASC"). | |||
| Scan(&itemConflicts).Error | |||
| if err != nil { | |||
| return nil, err | |||
| } | |||
| for _, conflict := range itemConflicts { | |||
| addConflict(conflict) | |||
| } | |||
| var activeBatches []activeBatchSelection | |||
| if err := DB.Model(&UserMigrationBatch{}). | |||
| Select("id, status, selection_mode, selection_payload"). | |||
| Where("status IN ?", activeStatuses). | |||
| Where("selection_mode = ?", "explicit_ids"). | |||
| Find(&activeBatches).Error; err != nil { | |||
| return nil, err | |||
| } | |||
| sourceIDSet := make(map[int]struct{}, len(sourceUserIDs)) | |||
| for _, id := range sourceUserIDs { | |||
| sourceIDSet[id] = struct{}{} | |||
| } | |||
| for _, batch := range activeBatches { | |||
| if batch.SelectionPayload == "" { | |||
| continue | |||
| } | |||
| var payload struct { | |||
| SourceUserIDs []int `json:"source_user_ids"` | |||
| } | |||
| if err := common.UnmarshalJsonStr(batch.SelectionPayload, &payload); err != nil { | |||
| return nil, err | |||
| } | |||
| for _, sourceUserID := range payload.SourceUserIDs { | |||
| if _, ok := sourceIDSet[sourceUserID]; !ok { | |||
| continue | |||
| } | |||
| addConflict(ActiveMigrationUserConflict{ | |||
| SourceUserID: sourceUserID, | |||
| BatchID: batch.ID, | |||
| BatchStatus: batch.Status, | |||
| }) | |||
| } | |||
| } | |||
| conflicts := make([]ActiveMigrationUserConflict, 0, len(conflictMap)) | |||
| for _, conflict := range conflictMap { | |||
| conflicts = append(conflicts, conflict) | |||
| } | |||
| slices.SortFunc(conflicts, func(a, b ActiveMigrationUserConflict) int { | |||
| if a.SourceUserID != b.SourceUserID { | |||
| return a.SourceUserID - b.SourceUserID | |||
| } | |||
| return a.BatchID - b.BatchID | |||
| }) | |||
| return conflicts, nil | |||
| } | |||
| func UpsertUserMigrationItem(item *UserMigrationItem) error { | |||
| var existing UserMigrationItem | |||
| result := DB.Where("batch_id = ? AND source_user_id = ?", item.BatchId, item.SourceUserId).Limit(1).Find(&existing) | |||
| if result.Error != nil { | |||
| return result.Error | |||
| } | |||
| if result.RowsAffected > 0 { | |||
| updates := map[string]any{ | |||
| "source_username": item.SourceUsername, | |||
| "source_email": item.SourceEmail, | |||
| "source_quota": item.SourceQuota, | |||
| "conflict_flags": item.ConflictFlags, | |||
| "status": item.Status, | |||
| "snapshot": item.Snapshot, | |||
| } | |||
| if existing.ResolutionStrategy == "" && item.ResolutionStrategy != "" { | |||
| updates["resolution_strategy"] = item.ResolutionStrategy | |||
| } | |||
| if existing.TargetUserId == 0 && item.TargetUserId != 0 { | |||
| updates["target_user_id"] = item.TargetUserId | |||
| } | |||
| item.Id = existing.Id | |||
| return DB.Model(&existing).Updates(updates).Error | |||
| } | |||
| return DB.Create(item).Error | |||
| } | |||
| func ListUserMigrationItemsByBatch(batchId int) ([]*UserMigrationItem, error) { | |||
| var items []*UserMigrationItem | |||
| err := DB.Where("batch_id = ?", batchId).Order("id asc").Find(&items).Error | |||
| return items, err | |||
| } | |||
| func ListExecutableUserMigrationItems(batchId int) ([]*UserMigrationItem, error) { | |||
| var items []*UserMigrationItem | |||
| err := DB.Where("batch_id = ? AND status IN ?", batchId, []string{ | |||
| UserMigrationItemStatusReady, | |||
| UserMigrationItemStatusMigrating, | |||
| UserMigrationItemStatusFailed, | |||
| }).Order("id asc").Find(&items).Error | |||
| return items, err | |||
| } | |||
| func MarkUserMigrationItemFailed(id int, errMsg string) error { | |||
| return DB.Model(&UserMigrationItem{}).Where("id = ?", id).Updates(map[string]any{ | |||
| "status": UserMigrationItemStatusFailed, | |||
| "error_message": errMsg, | |||
| }).Error | |||
| } | |||
| func MarkUserMigrationItemSkipped(id int) error { | |||
| return DB.Model(&UserMigrationItem{}).Where("id = ?", id). | |||
| Update("status", UserMigrationItemStatusSkipped).Error | |||
| } | |||
| func RefreshUserMigrationBatchStats(batchId int) error { | |||
| var total, conflicts, pendingScan int64 | |||
| DB.Model(&UserMigrationItem{}).Where("batch_id = ?", batchId).Count(&total) | |||
| DB.Model(&UserMigrationItem{}).Where("batch_id = ? AND status = ?", batchId, UserMigrationItemStatusConflict).Count(&conflicts) | |||
| DB.Model(&UserMigrationItem{}).Where("batch_id = ? AND status = ?", batchId, UserMigrationItemStatusPendingScan).Count(&pendingScan) | |||
| status := UserMigrationBatchStatusScanned | |||
| if conflicts == 0 && pendingScan == 0 && total > 0 { | |||
| status = UserMigrationBatchStatusReady | |||
| } | |||
| return DB.Model(&UserMigrationBatch{}).Where("id = ?", batchId).Updates(map[string]any{ | |||
| "total_users": int(total), | |||
| "conflict_users": int(conflicts), | |||
| "status": status, | |||
| }).Error | |||
| } | |||
| func RefreshUserMigrationBatchExecutionStats(batchId int) error { | |||
| var migrated, failed, pending, migrating int64 | |||
| DB.Model(&UserMigrationItem{}).Where("batch_id = ? AND status = ?", batchId, UserMigrationItemStatusMigrated).Count(&migrated) | |||
| DB.Model(&UserMigrationItem{}).Where("batch_id = ? AND status = ?", batchId, UserMigrationItemStatusFailed).Count(&failed) | |||
| DB.Model(&UserMigrationItem{}).Where("batch_id = ? AND status IN ?", batchId, []string{ | |||
| UserMigrationItemStatusReady, | |||
| UserMigrationItemStatusMigrating, | |||
| }).Count(&pending) | |||
| DB.Model(&UserMigrationItem{}).Where("batch_id = ? AND status = ?", batchId, UserMigrationItemStatusMigrating).Count(&migrating) | |||
| status := UserMigrationBatchStatusCompleted | |||
| if failed > 0 { | |||
| status = UserMigrationBatchStatusFailed | |||
| } | |||
| if pending > 0 { | |||
| status = UserMigrationBatchStatusRunning | |||
| } | |||
| if pending == migrating && migrating > 0 { | |||
| status = UserMigrationBatchStatusFailed | |||
| } | |||
| return DB.Model(&UserMigrationBatch{}).Where("id = ?", batchId).Updates(map[string]any{ | |||
| "migrated_users": int(migrated), | |||
| "failed_users": int(failed), | |||
| "status": status, | |||
| }).Error | |||
| } | |||
| func TryLockBatchForExecution(batchId int) (bool, error) { | |||
| result := DB.Model(&UserMigrationBatch{}). | |||
| Where("id = ? AND status IN ?", batchId, []string{ | |||
| UserMigrationBatchStatusReady, | |||
| UserMigrationBatchStatusFailed, | |||
| }). | |||
| Update("status", UserMigrationBatchStatusRunning) | |||
| return result.RowsAffected == 1, result.Error | |||
| } | |||
| func TryLockBatchForScan(batchId int) (bool, error) { | |||
| result := DB.Model(&UserMigrationBatch{}). | |||
| Where("id = ? AND status IN ?", batchId, []string{ | |||
| UserMigrationBatchStatusDraft, | |||
| UserMigrationBatchStatusScanned, | |||
| UserMigrationBatchStatusReady, | |||
| }). | |||
| Update("status", UserMigrationBatchStatusScanning) | |||
| return result.RowsAffected == 1, result.Error | |||
| } | |||
| func AtomicGrantQuotaIfNotExists(tx *gorm.DB, userId, delta int, token string) (bool, error) { | |||
| result := tx.Clauses(clause.OnConflict{DoNothing: true}).Create(&MigrationQuotaGrant{MigrationToken: token}) | |||
| if result.Error != nil { | |||
| return false, result.Error | |||
| } | |||
| if result.RowsAffected == 0 { | |||
| return true, nil | |||
| } | |||
| return false, tx.Model(&User{}).Where("id = ?", userId). | |||
| Update("quota", gorm.Expr("quota + ?", delta)).Error | |||
| } | |||
| @@ -0,0 +1,285 @@ | |||
| package model | |||
| import ( | |||
| "encoding/json" | |||
| "testing" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| func setupUserMigrationTestDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, err := db.DB() | |||
| require.NoError(t, err) | |||
| sqlDB.SetMaxOpenConns(1) | |||
| require.NoError(t, db.AutoMigrate( | |||
| &User{}, | |||
| &CustomOAuthProvider{}, | |||
| &UserOAuthBinding{}, | |||
| &UserMigrationBatch{}, | |||
| &UserMigrationItem{}, | |||
| &MigrationQuotaGrant{}, | |||
| )) | |||
| return db | |||
| } | |||
| func TestUserMigrationModels_AutoMigrate(t *testing.T) { | |||
| db := setupUserMigrationTestDB(t) | |||
| require.True(t, db.Migrator().HasTable(&UserMigrationBatch{})) | |||
| require.True(t, db.Migrator().HasTable(&UserMigrationItem{})) | |||
| require.True(t, db.Migrator().HasTable(&MigrationQuotaGrant{})) | |||
| } | |||
| func TestCreateUserMigrationBatch_DefaultStatus(t *testing.T) { | |||
| db := setupUserMigrationTestDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| batch := &UserMigrationBatch{ | |||
| Name: "ov-to-cn-wave-1", | |||
| SourceRegion: "overseas", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| } | |||
| require.NoError(t, CreateUserMigrationBatch(batch)) | |||
| require.Equal(t, UserMigrationBatchStatusDraft, batch.Status) | |||
| require.Equal(t, "all", batch.SelectionMode) | |||
| require.Greater(t, batch.Id, 0) | |||
| } | |||
| func TestCreateUserMigrationBatch_StoresExplicitIDsSelection(t *testing.T) { | |||
| db := setupUserMigrationTestDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| payload := map[string]any{"source_user_ids": []int{1001, 1002, 1003}} | |||
| payloadJSON, err := json.Marshal(payload) | |||
| require.NoError(t, err) | |||
| batch := &UserMigrationBatch{ | |||
| Name: "partial-wave", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| SelectionMode: "explicit_ids", | |||
| SelectionPayload: string(payloadJSON), | |||
| RequestedUserCount: 3, | |||
| OperatorId: 1, | |||
| } | |||
| require.NoError(t, CreateUserMigrationBatch(batch)) | |||
| var saved UserMigrationBatch | |||
| require.NoError(t, DB.First(&saved, batch.Id).Error) | |||
| require.Equal(t, "explicit_ids", saved.SelectionMode) | |||
| require.Equal(t, string(payloadJSON), saved.SelectionPayload) | |||
| require.Equal(t, 3, saved.RequestedUserCount) | |||
| require.Equal(t, 0, saved.MatchedUserCount) | |||
| require.Equal(t, 0, saved.ExcludedUserCount) | |||
| } | |||
| func TestUpsertMigrationItem_PreservesHumanSetStrategy(t *testing.T) { | |||
| db := setupUserMigrationTestDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| batch := &UserMigrationBatch{ | |||
| Name: "b1", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| } | |||
| require.NoError(t, CreateUserMigrationBatch(batch)) | |||
| first := &UserMigrationItem{ | |||
| BatchId: batch.Id, | |||
| SourceUserId: 10000001, | |||
| SourceUsername: "alice", | |||
| SourceEmail: "alice@example.com", | |||
| SourceQuota: 12345, | |||
| ResolutionStrategy: "", | |||
| Status: UserMigrationItemStatusConflict, | |||
| } | |||
| require.NoError(t, UpsertUserMigrationItem(first)) | |||
| require.NoError(t, DB.Model(&UserMigrationItem{}). | |||
| Where("batch_id = ? AND source_user_id = ?", batch.Id, 10000001). | |||
| Update("resolution_strategy", UserMigrationStrategyMergeExisting).Error) | |||
| second := &UserMigrationItem{ | |||
| BatchId: batch.Id, | |||
| SourceUserId: 10000001, | |||
| SourceUsername: "alice", | |||
| SourceEmail: "alice@example.com", | |||
| SourceQuota: 99999, | |||
| ResolutionStrategy: UserMigrationStrategyCreateNew, | |||
| Status: UserMigrationItemStatusReady, | |||
| } | |||
| require.NoError(t, UpsertUserMigrationItem(second)) | |||
| items, err := ListUserMigrationItemsByBatch(batch.Id) | |||
| require.NoError(t, err) | |||
| require.Len(t, items, 1) | |||
| require.Equal(t, UserMigrationStrategyMergeExisting, items[0].ResolutionStrategy) | |||
| require.Equal(t, UserMigrationItemStatusReady, items[0].Status) | |||
| require.Equal(t, 99999, items[0].SourceQuota) | |||
| } | |||
| func TestAtomicGrantQuotaIfNotExists_Idempotent(t *testing.T) { | |||
| db := setupUserMigrationTestDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| require.NoError(t, db.Create(&User{ | |||
| Id: 1, | |||
| Username: "cn-user", | |||
| Password: "hash", | |||
| AffCode: "AFF1", | |||
| Quota: 500, | |||
| }).Error) | |||
| var alreadyDone bool | |||
| err := db.Transaction(func(tx *gorm.DB) error { | |||
| var innerErr error | |||
| alreadyDone, innerErr = AtomicGrantQuotaIfNotExists(tx, 1, 200, "token-abc") | |||
| return innerErr | |||
| }) | |||
| require.NoError(t, err) | |||
| require.False(t, alreadyDone) | |||
| var user User | |||
| require.NoError(t, db.First(&user, 1).Error) | |||
| require.Equal(t, 700, user.Quota) | |||
| err = db.Transaction(func(tx *gorm.DB) error { | |||
| var innerErr error | |||
| alreadyDone, innerErr = AtomicGrantQuotaIfNotExists(tx, 1, 200, "token-abc") | |||
| return innerErr | |||
| }) | |||
| require.NoError(t, err) | |||
| require.True(t, alreadyDone) | |||
| require.NoError(t, db.First(&user, 1).Error) | |||
| require.Equal(t, 700, user.Quota) | |||
| } | |||
| func TestRefreshBatchStats_ReadyOnlyWhenNoPendingScan(t *testing.T) { | |||
| db := setupUserMigrationTestDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| batch := &UserMigrationBatch{Name: "stats-test", SourceRegion: "ov", TargetRegion: "cn", OperatorId: 1} | |||
| require.NoError(t, CreateUserMigrationBatch(batch)) | |||
| require.NoError(t, DB.Create(&UserMigrationItem{ | |||
| BatchId: batch.Id, | |||
| SourceUserId: 1, | |||
| SourceUsername: "u1", | |||
| Status: UserMigrationItemStatusReady, | |||
| }).Error) | |||
| require.NoError(t, DB.Create(&UserMigrationItem{ | |||
| BatchId: batch.Id, | |||
| SourceUserId: 2, | |||
| SourceUsername: "u2", | |||
| Status: UserMigrationItemStatusPendingScan, | |||
| }).Error) | |||
| require.NoError(t, RefreshUserMigrationBatchStats(batch.Id)) | |||
| var refreshed UserMigrationBatch | |||
| require.NoError(t, DB.First(&refreshed, batch.Id).Error) | |||
| require.Equal(t, UserMigrationBatchStatusScanned, refreshed.Status) | |||
| } | |||
| func TestRefreshUserMigrationBatchExecutionStats_MigratingLeakHandled(t *testing.T) { | |||
| db := setupUserMigrationTestDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| batch := &UserMigrationBatch{ | |||
| Name: "migrating-leak", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| Status: UserMigrationBatchStatusRunning, | |||
| } | |||
| require.NoError(t, CreateUserMigrationBatch(batch)) | |||
| require.NoError(t, DB.Create(&UserMigrationItem{ | |||
| BatchId: batch.Id, | |||
| SourceUserId: 101, | |||
| SourceUsername: "stuck-user", | |||
| Status: UserMigrationItemStatusMigrating, | |||
| }).Error) | |||
| require.NoError(t, RefreshUserMigrationBatchExecutionStats(batch.Id)) | |||
| var refreshed UserMigrationBatch | |||
| require.NoError(t, DB.First(&refreshed, batch.Id).Error) | |||
| require.Equal(t, UserMigrationBatchStatusFailed, refreshed.Status) | |||
| require.Equal(t, 0, refreshed.MigratedUsers) | |||
| require.Equal(t, 0, refreshed.FailedUsers) | |||
| } | |||
| func TestCancelUserMigrationBatch_ReleasesBatch(t *testing.T) { | |||
| db := setupUserMigrationTestDB(t) | |||
| orig := DB | |||
| DB = db | |||
| defer func() { | |||
| DB = orig | |||
| }() | |||
| batch := &UserMigrationBatch{ | |||
| Name: "cancel-batch", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| SelectionMode: "explicit_ids", | |||
| SelectionPayload: `{"source_user_ids":[1001,1002]}`, | |||
| OperatorId: 1, | |||
| Status: UserMigrationBatchStatusScanned, | |||
| } | |||
| require.NoError(t, CreateUserMigrationBatch(batch)) | |||
| conflicts, err := FindActiveMigrationUserConflicts([]int{1001, 1002}) | |||
| require.NoError(t, err) | |||
| require.Len(t, conflicts, 2) | |||
| cancelled, err := CancelUserMigrationBatch(batch.Id) | |||
| require.NoError(t, err) | |||
| require.True(t, cancelled) | |||
| var refreshed UserMigrationBatch | |||
| require.NoError(t, DB.First(&refreshed, batch.Id).Error) | |||
| require.Equal(t, UserMigrationBatchStatusCancelled, refreshed.Status) | |||
| conflicts, err = FindActiveMigrationUserConflicts([]int{1001, 1002}) | |||
| require.NoError(t, err) | |||
| require.Len(t, conflicts, 0) | |||
| } | |||
| @@ -10,9 +10,9 @@ import ( | |||
| // UserOAuthBinding stores the binding relationship between users and custom OAuth providers | |||
| type UserOAuthBinding struct { | |||
| Id int `json:"id" gorm:"primaryKey"` | |||
| UserId int `json:"user_id" gorm:"not null;uniqueIndex:ux_user_provider"` // User ID - one binding per user per provider | |||
| ProviderId int `json:"provider_id" gorm:"not null;uniqueIndex:ux_user_provider;uniqueIndex:ux_provider_userid"` // Custom OAuth provider ID | |||
| ProviderUserId string `json:"provider_user_id" gorm:"type:varchar(256);not null;uniqueIndex:ux_provider_userid"` // User ID from OAuth provider - one OAuth account per provider | |||
| UserId int `json:"user_id" gorm:"not null;uniqueIndex:ux_user_provider"` // User ID - one binding per user per provider | |||
| ProviderId int `json:"provider_id" gorm:"not null;uniqueIndex:ux_user_provider;uniqueIndex:ux_provider_userid"` // Custom OAuth provider ID | |||
| ProviderUserId string `json:"provider_user_id" gorm:"type:varchar(256);not null;uniqueIndex:ux_provider_userid"` // User ID from OAuth provider - one OAuth account per provider | |||
| CreatedAt time.Time `json:"created_at"` | |||
| } | |||
| @@ -40,15 +40,21 @@ func GetUserOAuthBinding(userId, providerId int) (*UserOAuthBinding, error) { | |||
| // GetUserByOAuthBinding finds a user by provider ID and provider user ID | |||
| func GetUserByOAuthBinding(providerId int, providerUserId string) (*User, error) { | |||
| var binding UserOAuthBinding | |||
| err := DB.Where("provider_id = ? AND provider_user_id = ?", providerId, providerUserId).First(&binding).Error | |||
| if err != nil { | |||
| return nil, err | |||
| result := DB.Where("provider_id = ? AND provider_user_id = ?", providerId, providerUserId).Limit(1).Find(&binding) | |||
| if result.Error != nil { | |||
| return nil, result.Error | |||
| } | |||
| if result.RowsAffected == 0 { | |||
| return nil, gorm.ErrRecordNotFound | |||
| } | |||
| var user User | |||
| err = DB.First(&user, binding.UserId).Error | |||
| if err != nil { | |||
| return nil, err | |||
| userResult := DB.Where("id = ?", binding.UserId).Limit(1).Find(&user) | |||
| if userResult.Error != nil { | |||
| return nil, userResult.Error | |||
| } | |||
| if userResult.RowsAffected == 0 { | |||
| return nil, gorm.ErrRecordNotFound | |||
| } | |||
| return &user, nil | |||
| } | |||
| @@ -104,6 +110,36 @@ func CreateUserOAuthBindingWithTx(tx *gorm.DB, binding *UserOAuthBinding) error | |||
| return tx.Create(binding).Error | |||
| } | |||
| func CopyOAuthBindingsIfMissing(userId int, bindings []*UserOAuthBinding) error { | |||
| return CopyOAuthBindingsIfMissingTx(DB, userId, bindings) | |||
| } | |||
| func CopyOAuthBindingsIfMissingTx(tx *gorm.DB, userId int, bindings []*UserOAuthBinding) error { | |||
| for _, binding := range bindings { | |||
| if binding == nil { | |||
| continue | |||
| } | |||
| var existing UserOAuthBinding | |||
| result := tx.Where("user_id = ? AND provider_id = ?", userId, binding.ProviderId).Limit(1).Find(&existing) | |||
| if result.Error != nil { | |||
| return result.Error | |||
| } | |||
| if result.RowsAffected > 0 { | |||
| continue | |||
| } | |||
| newBinding := &UserOAuthBinding{ | |||
| UserId: userId, | |||
| ProviderId: binding.ProviderId, | |||
| ProviderUserId: binding.ProviderUserId, | |||
| } | |||
| if err := CreateUserOAuthBindingWithTx(tx, newBinding); err != nil { | |||
| return err | |||
| } | |||
| } | |||
| return nil | |||
| } | |||
| // UpdateUserOAuthBinding updates an existing OAuth binding (e.g., rebind to different OAuth account) | |||
| func UpdateUserOAuthBinding(userId, providerId int, newProviderUserId string) error { | |||
| // Check if the new provider user ID is already taken by another user | |||
| @@ -18,12 +18,14 @@ import ( | |||
| "github.com/QuantumNous/new-api/relay/constant" | |||
| "github.com/QuantumNous/new-api/relay/helper" | |||
| "github.com/QuantumNous/new-api/service" | |||
| "github.com/QuantumNous/new-api/setting/model_setting" | |||
| "github.com/QuantumNous/new-api/setting/operation_setting" | |||
| "github.com/QuantumNous/new-api/types" | |||
| "github.com/bytedance/gopkg/util/gopool" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/gorilla/websocket" | |||
| "github.com/tidwall/sjson" | |||
| ) | |||
| func SetupApiRequestHeader(info *common.RelayInfo, c *gin.Context, req *http.Header) { | |||
| @@ -277,6 +279,36 @@ func applyHeaderOverrideToRequest(req *http.Request, headerOverride map[string]s | |||
| } | |||
| } | |||
| func needsPassthroughModelMapping(info *common.RelayInfo) bool { | |||
| if info == nil || info.ChannelMeta == nil { | |||
| return false | |||
| } | |||
| if !model_setting.GetGlobalSettings().PassthroughApplyModelMapping { | |||
| return false | |||
| } | |||
| if !model_setting.GetGlobalSettings().PassThroughRequestEnabled && !info.ChannelMeta.ChannelSetting.PassThroughBodyEnabled { | |||
| return false | |||
| } | |||
| if !info.ChannelMeta.IsModelMapped || strings.TrimSpace(info.ChannelMeta.UpstreamModelName) == "" { | |||
| return false | |||
| } | |||
| return true | |||
| } | |||
| func applyPassthroughModelMapping(body []byte, info *common.RelayInfo) []byte { | |||
| if len(body) == 0 { | |||
| return body | |||
| } | |||
| if common2.GetJsonType(body) != "object" { | |||
| return body | |||
| } | |||
| modified, err := sjson.SetBytes(body, "model", info.ChannelMeta.UpstreamModelName) | |||
| if err != nil { | |||
| return body | |||
| } | |||
| return modified | |||
| } | |||
| func DoApiRequest(a Adaptor, c *gin.Context, info *common.RelayInfo, requestBody io.Reader) (*http.Response, error) { | |||
| fullRequestURL, err := a.GetRequestURL(info) | |||
| if err != nil { | |||
| @@ -285,7 +317,15 @@ func DoApiRequest(a Adaptor, c *gin.Context, info *common.RelayInfo, requestBody | |||
| if common2.DebugEnabled { | |||
| common2.SysLog("fullRequestURL: " + fullRequestURL) | |||
| } | |||
| req, err := http.NewRequest(c.Request.Method, fullRequestURL, requestBody) | |||
| var bodyReader io.Reader = requestBody | |||
| if needsPassthroughModelMapping(info) && requestBody != nil { | |||
| bodyBytes, readErr := io.ReadAll(requestBody) | |||
| if readErr != nil { | |||
| return nil, fmt.Errorf("read request body failed: %w", readErr) | |||
| } | |||
| bodyReader = bytes.NewReader(applyPassthroughModelMapping(bodyBytes, info)) | |||
| } | |||
| req, err := http.NewRequest(c.Request.Method, fullRequestURL, bodyReader) | |||
| if err != nil { | |||
| return nil, fmt.Errorf("new request failed: %w", err) | |||
| } | |||
| @@ -6,10 +6,69 @@ import ( | |||
| "testing" | |||
| relaycommon "github.com/QuantumNous/new-api/relay/common" | |||
| "github.com/QuantumNous/new-api/setting/model_setting" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/stretchr/testify/assert" | |||
| "github.com/stretchr/testify/require" | |||
| ) | |||
| func TestPassthroughModelMapping_AppliedWhenEnabled(t *testing.T) { | |||
| origRequestEnabled := model_setting.GetGlobalSettings().PassThroughRequestEnabled | |||
| origApplyMapping := model_setting.GetGlobalSettings().PassthroughApplyModelMapping | |||
| defer func() { | |||
| model_setting.GetGlobalSettings().PassThroughRequestEnabled = origRequestEnabled | |||
| model_setting.GetGlobalSettings().PassthroughApplyModelMapping = origApplyMapping | |||
| }() | |||
| model_setting.GetGlobalSettings().PassThroughRequestEnabled = true | |||
| model_setting.GetGlobalSettings().PassthroughApplyModelMapping = true | |||
| info := &relaycommon.RelayInfo{ChannelMeta: &relaycommon.ChannelMeta{IsModelMapped: true, UpstreamModelName: "gpt-4.1"}} | |||
| body := []byte(`{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}]}`) | |||
| assert.True(t, needsPassthroughModelMapping(info)) | |||
| got := applyPassthroughModelMapping(body, info) | |||
| assert.JSONEq(t, `{"model":"gpt-4.1","messages":[{"role":"user","content":"hi"}]}`, string(got)) | |||
| } | |||
| func TestPassthroughModelMapping_SkippedWhenNotMapped(t *testing.T) { | |||
| origRequestEnabled := model_setting.GetGlobalSettings().PassThroughRequestEnabled | |||
| origApplyMapping := model_setting.GetGlobalSettings().PassthroughApplyModelMapping | |||
| defer func() { | |||
| model_setting.GetGlobalSettings().PassThroughRequestEnabled = origRequestEnabled | |||
| model_setting.GetGlobalSettings().PassthroughApplyModelMapping = origApplyMapping | |||
| }() | |||
| model_setting.GetGlobalSettings().PassThroughRequestEnabled = true | |||
| model_setting.GetGlobalSettings().PassthroughApplyModelMapping = true | |||
| info := &relaycommon.RelayInfo{ChannelMeta: &relaycommon.ChannelMeta{IsModelMapped: false, UpstreamModelName: "gpt-4.1"}} | |||
| assert.False(t, needsPassthroughModelMapping(info)) | |||
| } | |||
| func TestPassthroughModelMapping_SkippedWhenPassthroughOff(t *testing.T) { | |||
| origRequestEnabled := model_setting.GetGlobalSettings().PassThroughRequestEnabled | |||
| origApplyMapping := model_setting.GetGlobalSettings().PassthroughApplyModelMapping | |||
| defer func() { | |||
| model_setting.GetGlobalSettings().PassThroughRequestEnabled = origRequestEnabled | |||
| model_setting.GetGlobalSettings().PassthroughApplyModelMapping = origApplyMapping | |||
| }() | |||
| model_setting.GetGlobalSettings().PassThroughRequestEnabled = false | |||
| model_setting.GetGlobalSettings().PassthroughApplyModelMapping = true | |||
| info := &relaycommon.RelayInfo{ChannelMeta: &relaycommon.ChannelMeta{IsModelMapped: true, UpstreamModelName: "gpt-4.1"}} | |||
| assert.False(t, needsPassthroughModelMapping(info)) | |||
| } | |||
| func TestPassthroughModelMapping_NonJSONKeepsOriginal(t *testing.T) { | |||
| body := []byte(`model=gpt-4o`) | |||
| info := &relaycommon.RelayInfo{ChannelMeta: &relaycommon.ChannelMeta{IsModelMapped: true, UpstreamModelName: "gpt-4.1"}} | |||
| got := applyPassthroughModelMapping(body, info) | |||
| assert.Equal(t, body, got) | |||
| } | |||
| func TestProcessHeaderOverride_ChannelTestSkipsPassthroughRules(t *testing.T) { | |||
| t.Parallel() | |||
| @@ -163,7 +163,7 @@ func RequestOpenAI2ClaudeMessage(c *gin.Context, textRequest dto.GeneralOpenAIRe | |||
| // BudgetTokens 为 max_tokens 的 80% | |||
| claudeRequest.Thinking = &dto.Thinking{ | |||
| Type: "enabled", | |||
| Type: "adaptive", | |||
| BudgetTokens: common.GetPointer[int](int(float64(claudeRequest.MaxTokens) * model_setting.GetClaudeSettings().ThinkingAdapterBudgetTokensPercentage)), | |||
| } | |||
| // TODO: 临时处理 | |||
| @@ -179,17 +179,17 @@ func RequestOpenAI2ClaudeMessage(c *gin.Context, textRequest dto.GeneralOpenAIRe | |||
| switch textRequest.ReasoningEffort { | |||
| case "low": | |||
| claudeRequest.Thinking = &dto.Thinking{ | |||
| Type: "enabled", | |||
| Type: "adaptive", | |||
| BudgetTokens: common.GetPointer[int](1280), | |||
| } | |||
| case "medium": | |||
| claudeRequest.Thinking = &dto.Thinking{ | |||
| Type: "enabled", | |||
| Type: "adaptive", | |||
| BudgetTokens: common.GetPointer[int](2048), | |||
| } | |||
| case "high": | |||
| claudeRequest.Thinking = &dto.Thinking{ | |||
| Type: "enabled", | |||
| Type: "adaptive", | |||
| BudgetTokens: common.GetPointer[int](4096), | |||
| } | |||
| } | |||
| @@ -205,7 +205,7 @@ func RequestOpenAI2ClaudeMessage(c *gin.Context, textRequest dto.GeneralOpenAIRe | |||
| budgetTokens := reasoning.MaxTokens | |||
| if budgetTokens > 0 { | |||
| claudeRequest.Thinking = &dto.Thinking{ | |||
| Type: "enabled", | |||
| Type: "adaptive", | |||
| BudgetTokens: &budgetTokens, | |||
| } | |||
| } | |||
| @@ -409,7 +409,7 @@ func OaiResponsesToChatStreamHandler(c *gin.Context, info *relaycommon.RelayInfo | |||
| toolCallNameByID[callID] = name | |||
| } | |||
| newArgs := streamResp.Item.Arguments | |||
| newArgs := streamResp.Item.GetArguments() | |||
| prevArgs := toolCallArgsByID[callID] | |||
| argsDelta := "" | |||
| if newArgs != "" { | |||
| @@ -19,6 +19,8 @@ import ( | |||
| "github.com/QuantumNous/new-api/types" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/tidwall/gjson" | |||
| "github.com/tidwall/sjson" | |||
| ) | |||
| func ClaudeHelper(c *gin.Context, info *relaycommon.RelayInfo) (newAPIError *types.NewAPIError) { | |||
| @@ -71,7 +73,7 @@ func ClaudeHelper(c *gin.Context, info *relaycommon.RelayInfo) (newAPIError *typ | |||
| // BudgetTokens 为 max_tokens 的 80% | |||
| request.Thinking = &dto.Thinking{ | |||
| Type: "enabled", | |||
| Type: "adaptive", | |||
| BudgetTokens: common.GetPointer[int](int(float64(request.MaxTokens) * model_setting.GetClaudeSettings().ThinkingAdapterBudgetTokensPercentage)), | |||
| } | |||
| // TODO: 临时处理 | |||
| @@ -127,13 +129,31 @@ func ClaudeHelper(c *gin.Context, info *relaycommon.RelayInfo) (newAPIError *typ | |||
| return nil | |||
| } | |||
| // Fix thinking.type for models that only support "adaptive" (e.g. claude-sonnet-4-6, claude-opus-4-6). | |||
| // Anthropic returns 400 if thinking.type="enabled" is sent to these models. | |||
| if request.Thinking != nil && request.Thinking.Type == "enabled" { | |||
| request.Thinking.Type = "adaptive" | |||
| } | |||
| var requestBody io.Reader | |||
| if model_setting.GetGlobalSettings().PassThroughRequestEnabled || info.ChannelSetting.PassThroughBodyEnabled { | |||
| storage, err := common.GetBodyStorage(c) | |||
| if err != nil { | |||
| return types.NewErrorWithStatusCode(err, types.ErrorCodeReadRequestBodyFailed, http.StatusBadRequest, types.ErrOptionWithSkipRetry()) | |||
| } | |||
| requestBody = common.ReaderOnly(storage) | |||
| // In passthrough mode, fix thinking.type in the raw JSON body. | |||
| if request.Thinking != nil && request.Thinking.Type == "adaptive" { | |||
| bodyBytes, readErr := io.ReadAll(storage) | |||
| if readErr != nil { | |||
| return types.NewErrorWithStatusCode(readErr, types.ErrorCodeReadRequestBodyFailed, http.StatusBadRequest, types.ErrOptionWithSkipRetry()) | |||
| } | |||
| if gjson.GetBytes(bodyBytes, "thinking.type").String() == "enabled" { | |||
| bodyBytes, _ = sjson.SetBytes(bodyBytes, "thinking.type", "adaptive") | |||
| } | |||
| requestBody = bytes.NewBuffer(bodyBytes) | |||
| } else { | |||
| requestBody = common.ReaderOnly(storage) | |||
| } | |||
| } else { | |||
| convertedRequest, err := adaptor.ConvertClaudeRequest(c, info, request) | |||
| if err != nil { | |||
| @@ -5,10 +5,12 @@ import ( | |||
| "fmt" | |||
| "io" | |||
| "net/http" | |||
| "strconv" | |||
| "strings" | |||
| "time" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/common/metrics" | |||
| "github.com/QuantumNous/new-api/constant" | |||
| "github.com/QuantumNous/new-api/dto" | |||
| "github.com/QuantumNous/new-api/logger" | |||
| @@ -285,7 +287,7 @@ func postConsumeQuota(ctx *gin.Context, relayInfo *relaycommon.RelayInfo, usage | |||
| dCachedCreationRatio := decimal.NewFromFloat(cachedCreationRatio) | |||
| dQuotaPerUnit := decimal.NewFromFloat(common.QuotaPerUnit) | |||
| ratio := dModelRatio.Mul(dGroupRatio).Mul(decimal.NewFromFloat(relayInfo.PriceData.UserChannelRatio)) | |||
| ratio := dModelRatio.Mul(dGroupRatio) | |||
| // openai web search 工具计费 | |||
| var dWebSearchQuota decimal.Decimal | |||
| @@ -399,7 +401,7 @@ func postConsumeQuota(ctx *gin.Context, relayInfo *relaycommon.RelayInfo, usage | |||
| quotaCalculateDecimal = decimal.NewFromInt(1) | |||
| } | |||
| } else { | |||
| quotaCalculateDecimal = dModelPrice.Mul(dQuotaPerUnit).Mul(dGroupRatio).Mul(decimal.NewFromFloat(relayInfo.PriceData.UserChannelRatio)) | |||
| quotaCalculateDecimal = dModelPrice.Mul(dQuotaPerUnit).Mul(dGroupRatio) | |||
| } | |||
| // 添加 responses tools call 调用的配额 | |||
| quotaCalculateDecimal = quotaCalculateDecimal.Add(dWebSearchQuota) | |||
| @@ -517,4 +519,31 @@ func postConsumeQuota(ctx *gin.Context, relayInfo *relaycommon.RelayInfo, usage | |||
| Group: relayInfo.UsingGroup, | |||
| Other: other, | |||
| }) | |||
| // Prometheus metrics 采集 | |||
| if metrics.IsEnabled() { | |||
| channelIdStr := strconv.Itoa(relayInfo.ChannelId) | |||
| modelName := relayInfo.OriginModelName | |||
| m := metrics.GetMetrics() | |||
| if promptTokens > 0 { | |||
| m.TokensTotal.WithLabelValues(channelIdStr, modelName, "prompt").Add(float64(promptTokens)) | |||
| } | |||
| if completionTokens > 0 { | |||
| m.TokensTotal.WithLabelValues(channelIdStr, modelName, "completion").Add(float64(completionTokens)) | |||
| } | |||
| if cacheTokens > 0 { | |||
| m.TokensTotal.WithLabelValues(channelIdStr, modelName, "cache").Add(float64(cacheTokens)) | |||
| } | |||
| m.QuotaConsumedTotal.WithLabelValues(channelIdStr, modelName, relayInfo.BillingSource).Add(float64(quota)) | |||
| upstreamDuration := time.Since(relayInfo.StartTime).Seconds() | |||
| m.UpstreamDuration.WithLabelValues(channelIdStr, modelName).Observe(upstreamDuration) | |||
| if relayInfo.IsStream && relayInfo.FirstResponseTime.After(relayInfo.StartTime) { | |||
| firstTokenDuration := relayInfo.FirstResponseTime.Sub(relayInfo.StartTime).Seconds() | |||
| m.FirstTokenDuration.WithLabelValues(channelIdStr, modelName).Observe(firstTokenDuration) | |||
| } | |||
| } | |||
| } | |||
| @@ -4,9 +4,7 @@ import ( | |||
| "fmt" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/constant" | |||
| "github.com/QuantumNous/new-api/logger" | |||
| "github.com/QuantumNous/new-api/model" | |||
| relaycommon "github.com/QuantumNous/new-api/relay/common" | |||
| "github.com/QuantumNous/new-api/setting/operation_setting" | |||
| "github.com/QuantumNous/new-api/setting/ratio_setting" | |||
| @@ -18,26 +16,22 @@ import ( | |||
| // HandleGroupRatio checks for "auto_group" in the context and updates the group ratio and relayInfo.UsingGroup if present | |||
| func HandleGroupRatio(ctx *gin.Context, relayInfo *relaycommon.RelayInfo) types.GroupRatioInfo { | |||
| groupRatioInfo := types.GroupRatioInfo{ | |||
| GroupRatio: 1.0, // default ratio | |||
| GroupRatio: 1.0, | |||
| GroupSpecialRatio: -1, | |||
| } | |||
| // check auto group | |||
| autoGroup, exists := ctx.Get("auto_group") | |||
| if exists { | |||
| logger.LogDebug(ctx, fmt.Sprintf("final group: %s", autoGroup)) | |||
| relayInfo.UsingGroup = autoGroup.(string) | |||
| } | |||
| // check user group special ratio | |||
| userGroupRatio, ok := ratio_setting.GetGroupGroupRatio(relayInfo.UserGroup, relayInfo.UsingGroup) | |||
| if ok { | |||
| // user group special ratio | |||
| groupRatioInfo.GroupSpecialRatio = userGroupRatio | |||
| groupRatioInfo.GroupRatio = userGroupRatio | |||
| groupRatioInfo.HasSpecialRatio = true | |||
| } else { | |||
| // normal group ratio | |||
| groupRatioInfo.GroupRatio = ratio_setting.GetGroupRatio(relayInfo.UsingGroup) | |||
| } | |||
| @@ -49,39 +43,8 @@ func ModelPriceHelper(c *gin.Context, info *relaycommon.RelayInfo, promptTokens | |||
| var usePrice bool | |||
| var modelRatio float64 | |||
| var completionRatio float64 | |||
| var channelPricingFound bool | |||
| var cacheRatio, cacheCreationRatio, imageRatio, audioRatio, audioCompletionRatio float64 | |||
| // 尝试获取渠道定价(优先于全局定价) | |||
| // ChannelMeta 在 InitChannelMeta 之前为 nil,但 Distribute 已将 channelId 写入 context | |||
| channelId := 0 | |||
| if info != nil && info.ChannelMeta != nil && info.ChannelId > 0 { | |||
| channelId = info.ChannelId | |||
| } else { | |||
| channelId = common.GetContextKeyInt(c, constant.ContextKeyChannelId) | |||
| } | |||
| if channelId > 0 { | |||
| cp, found := model.GetEffectivePricing(info.OriginModelName, channelId) | |||
| if found && cp != nil { | |||
| modelRatio = cp.ModelRatio | |||
| completionRatio = cp.CompletionRatio | |||
| modelPrice = cp.ModelPrice | |||
| usePrice = cp.QuotaType == model.QuotaTypeByCall | |||
| channelPricingFound = true | |||
| // 渠道定价的扩展比率(非零值直接使用) | |||
| cacheRatio = cp.CacheRatio | |||
| cacheCreationRatio = cp.CacheCreationRatio | |||
| imageRatio = cp.ImageRatio | |||
| audioRatio = cp.AudioRatio | |||
| audioCompletionRatio = cp.AudioCompletionRatio | |||
| } | |||
| } | |||
| // 如果没有渠道定价,使用全局定价 | |||
| if !channelPricingFound { | |||
| modelPrice, usePrice = ratio_setting.GetModelPrice(info.OriginModelName, false) | |||
| } | |||
| modelPrice, usePrice = ratio_setting.GetModelPrice(info.OriginModelName, false) | |||
| groupRatioInfo := HandleGroupRatio(c, info) | |||
| var preConsumedQuota int | |||
| @@ -91,22 +54,21 @@ func ModelPriceHelper(c *gin.Context, info *relaycommon.RelayInfo, promptTokens | |||
| if meta.MaxTokens != 0 { | |||
| preConsumedTokens += meta.MaxTokens | |||
| } | |||
| // 如果没有找到渠道定价,获取全局 modelRatio 和 completionRatio | |||
| if !channelPricingFound { | |||
| var success bool | |||
| var matchName string | |||
| modelRatio, success, matchName = ratio_setting.GetModelRatio(info.OriginModelName) | |||
| if !success { | |||
| acceptUnsetRatio := false | |||
| if info.UserSetting.AcceptUnsetRatioModel { | |||
| acceptUnsetRatio = true | |||
| } | |||
| if !acceptUnsetRatio { | |||
| return types.PriceData{}, fmt.Errorf("模型 %s 倍率或价格未配置,请联系管理员设置或开启自用模式;Model %s ratio or price not set, please set or start self-use mode", matchName, matchName) | |||
| } | |||
| var success bool | |||
| var matchName string | |||
| modelRatio, success, matchName = ratio_setting.GetModelRatio(info.OriginModelName) | |||
| if !success { | |||
| acceptUnsetRatio := false | |||
| if info.UserSetting.AcceptUnsetRatioModel { | |||
| acceptUnsetRatio = true | |||
| } | |||
| if !acceptUnsetRatio { | |||
| return types.PriceData{}, fmt.Errorf("模型 %s 倍率或价格未配置,请联系管理员设置或开启自用模式;Model %s ratio or price not set, please set or start self-use mode", matchName, matchName) | |||
| } | |||
| completionRatio = ratio_setting.GetCompletionRatio(info.OriginModelName) | |||
| } | |||
| completionRatio = ratio_setting.GetCompletionRatio(info.OriginModelName) | |||
| ratio := modelRatio * groupRatioInfo.GroupRatio | |||
| preConsumedQuota = int(float64(preConsumedTokens) * ratio) | |||
| } else { | |||
| @@ -116,27 +78,13 @@ func ModelPriceHelper(c *gin.Context, info *relaycommon.RelayInfo, promptTokens | |||
| preConsumedQuota = int(modelPrice * common.QuotaPerUnit * groupRatioInfo.GroupRatio) | |||
| } | |||
| // 全局比率作为回退(仅当渠道未设置、值为 0 时生效) | |||
| // 必须放在 usePrice 判断之外,因为 UpdatePriceDataForChannelPricing 可能改变 UsePrice | |||
| if cacheRatio == 0 { | |||
| cacheRatio, _ = ratio_setting.GetCacheRatio(info.OriginModelName) | |||
| } | |||
| if cacheCreationRatio == 0 { | |||
| cacheCreationRatio, _ = ratio_setting.GetCreateCacheRatio(info.OriginModelName) | |||
| } | |||
| if imageRatio == 0 { | |||
| imageRatio, _ = ratio_setting.GetImageRatio(info.OriginModelName) | |||
| } | |||
| if audioRatio == 0 { | |||
| audioRatio = ratio_setting.GetAudioRatio(info.OriginModelName) | |||
| } | |||
| if audioCompletionRatio == 0 { | |||
| audioCompletionRatio = ratio_setting.GetAudioCompletionRatio(info.OriginModelName) | |||
| } | |||
| cacheRatio, _ := ratio_setting.GetCacheRatio(info.OriginModelName) | |||
| cacheCreationRatio, _ := ratio_setting.GetCreateCacheRatio(info.OriginModelName) | |||
| imageRatio, _ := ratio_setting.GetImageRatio(info.OriginModelName) | |||
| audioRatio := ratio_setting.GetAudioRatio(info.OriginModelName) | |||
| audioCompletionRatio := ratio_setting.GetAudioCompletionRatio(info.OriginModelName) | |||
| // check if free model pre-consume is disabled | |||
| if !operation_setting.GetQuotaSetting().EnableFreeModelPreConsume { | |||
| // if model price or ratio is 0, do not pre-consume quota | |||
| if groupRatioInfo.GroupRatio == 0 { | |||
| preConsumedQuota = 0 | |||
| freeModel = true | |||
| @@ -145,11 +93,9 @@ func ModelPriceHelper(c *gin.Context, info *relaycommon.RelayInfo, promptTokens | |||
| preConsumedQuota = 0 | |||
| freeModel = true | |||
| } | |||
| } else { | |||
| if modelRatio == 0 { | |||
| preConsumedQuota = 0 | |||
| freeModel = true | |||
| } | |||
| } else if modelRatio == 0 { | |||
| preConsumedQuota = 0 | |||
| freeModel = true | |||
| } | |||
| } | |||
| @@ -168,7 +114,6 @@ func ModelPriceHelper(c *gin.Context, info *relaycommon.RelayInfo, promptTokens | |||
| CacheCreationRatio: cacheCreationRatio, | |||
| CacheCreation5mRatio: cacheCreationRatio, | |||
| CacheCreation1hRatio: cacheCreationRatio * types.ClaudeCacheCreation1hMultiplier, | |||
| UserChannelRatio: 1.0, | |||
| } | |||
| if common.DebugEnabled { | |||
| @@ -183,7 +128,6 @@ func ModelPriceHelperPerCall(c *gin.Context, info *relaycommon.RelayInfo) types. | |||
| groupRatioInfo := HandleGroupRatio(c, info) | |||
| modelPrice, success := ratio_setting.GetModelPrice(info.OriginModelName, true) | |||
| // 如果没有配置价格,则使用默认价格 | |||
| if !success { | |||
| defaultPrice, ok := ratio_setting.GetDefaultModelPriceMap()[info.OriginModelName] | |||
| if !ok { | |||
| @@ -194,7 +138,6 @@ func ModelPriceHelperPerCall(c *gin.Context, info *relaycommon.RelayInfo) types. | |||
| } | |||
| quota := int(modelPrice * common.QuotaPerUnit * groupRatioInfo.GroupRatio) | |||
| // 免费模型检测(与 ModelPriceHelper 对齐) | |||
| freeModel := false | |||
| if !operation_setting.GetQuotaSetting().EnableFreeModelPreConsume { | |||
| if groupRatioInfo.GroupRatio == 0 || modelPrice == 0 { | |||
| @@ -204,11 +147,10 @@ func ModelPriceHelperPerCall(c *gin.Context, info *relaycommon.RelayInfo) types. | |||
| } | |||
| priceData := types.PriceData{ | |||
| FreeModel: freeModel, | |||
| ModelPrice: modelPrice, | |||
| Quota: quota, | |||
| GroupRatioInfo: groupRatioInfo, | |||
| UserChannelRatio: 1.0, | |||
| FreeModel: freeModel, | |||
| ModelPrice: modelPrice, | |||
| Quota: quota, | |||
| GroupRatioInfo: groupRatioInfo, | |||
| } | |||
| return priceData | |||
| } | |||
| @@ -224,64 +166,3 @@ func ContainPriceOrRatio(modelName string) bool { | |||
| } | |||
| return false | |||
| } | |||
| // UpdatePriceDataForChannelPricing 在渠道选择后更新 PriceData 以使用渠道定价 | |||
| // ModelPriceHelper 在渠道选择之前被调用,此时 ChannelMeta 为 nil,导致渠道定价无法使用。 | |||
| // 需要在渠道选择后调用此函数更新价格数据,用于后续的日志记录和计费计算。 | |||
| func UpdatePriceDataForChannelPricing(c *gin.Context, info *relaycommon.RelayInfo, channelId int) { | |||
| if info == nil || channelId <= 0 { | |||
| return | |||
| } | |||
| cp, found := model.GetEffectivePricing(info.OriginModelName, channelId) | |||
| if found { | |||
| info.PriceData.ModelRatio = cp.ModelRatio | |||
| info.PriceData.CompletionRatio = cp.CompletionRatio | |||
| info.PriceData.UsePrice = cp.QuotaType == model.QuotaTypeByCall | |||
| if info.PriceData.UsePrice { | |||
| info.PriceData.ModelPrice = cp.ModelPrice | |||
| } else { | |||
| info.PriceData.ModelPrice = -1 | |||
| } | |||
| info.PriceData.ApplyChannelPricingRatios(cp.CacheRatio, cp.CacheCreationRatio, cp.ImageRatio, cp.AudioRatio, cp.AudioCompletionRatio) | |||
| if info.PriceData.UsePrice { | |||
| info.PriceData.QuotaToPreConsume = int( | |||
| cp.ModelPrice * common.QuotaPerUnit * info.PriceData.GroupRatioInfo.GroupRatio) | |||
| } else { | |||
| estimateTokens := info.GetEstimatePromptTokens() | |||
| if estimateTokens > 0 { | |||
| ratio := cp.ModelRatio * info.PriceData.GroupRatioInfo.GroupRatio | |||
| info.PriceData.QuotaToPreConsume = int(float64(estimateTokens) * ratio) | |||
| } | |||
| } | |||
| if common.DebugEnabled { | |||
| println(fmt.Sprintf("[ChannelPricing] updatePriceData: model=%s channel=%d modelRatio=%.4f completionRatio=%.4f cacheRatio=%.4f imageRatio=%.4f audioRatio=%.4f", | |||
| info.OriginModelName, channelId, cp.ModelRatio, cp.CompletionRatio, cp.CacheRatio, cp.ImageRatio, cp.AudioRatio)) | |||
| } | |||
| } | |||
| // 应用用户-模型-渠道倍率(始终执行,不依赖渠道定价是否存在) | |||
| ApplyUserChannelRatio(c, info, channelId) | |||
| } | |||
| // ApplyUserChannelRatio 在渠道选定后应用用户倍率 | |||
| func ApplyUserChannelRatio(c *gin.Context, info *relaycommon.RelayInfo, channelId int) { | |||
| if info == nil || channelId <= 0 { | |||
| return | |||
| } | |||
| userId := common.GetContextKeyInt(c, constant.ContextKeyUserId) | |||
| if userId <= 0 { | |||
| return | |||
| } | |||
| userRatio := model.GetUserChannelRatio(userId, info.OriginModelName, channelId) | |||
| if userRatio == 1.0 { | |||
| return | |||
| } | |||
| info.PriceData.UserChannelRatio = userRatio | |||
| if info.PriceData.QuotaToPreConsume > 0 { | |||
| info.PriceData.QuotaToPreConsume = int(float64(info.PriceData.QuotaToPreConsume) * userRatio) | |||
| } | |||
| } | |||
| @@ -9,10 +9,11 @@ import ( | |||
| "github.com/QuantumNous/new-api/dto" | |||
| "github.com/QuantumNous/new-api/model" | |||
| relaycommon "github.com/QuantumNous/new-api/relay/common" | |||
| "github.com/QuantumNous/new-api/setting/operation_setting" | |||
| "github.com/QuantumNous/new-api/setting/ratio_setting" | |||
| "github.com/QuantumNous/new-api/types" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/assert" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| @@ -23,54 +24,42 @@ const testModelUsePriceSwitch = "test-useprice-switch-model" | |||
| func setupUsePriceSwitchTest(t *testing.T) { | |||
| t.Helper() | |||
| // 初始化 SQLite 内存数据库 | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, _ := db.DB() | |||
| sqlDB, err := db.DB() | |||
| require.NoError(t, err) | |||
| sqlDB.SetMaxOpenConns(1) | |||
| origDB := model.DB | |||
| origUsingSQLite := common.UsingSQLite | |||
| origRedisEnabled := common.RedisEnabled | |||
| model.DB = db | |||
| common.UsingSQLite = true | |||
| common.RedisEnabled = false | |||
| require.NoError(t, db.AutoMigrate(&model.ChannelPricing{})) | |||
| // 全局定价:按次计费(modelPrice 存在 → usePrice=true) | |||
| require.NoError(t, ratio_setting.UpdateModelPriceByJSONString(`{"`+testModelUsePriceSwitch+`":0.5}`)) | |||
| // 全局 modelRatio 也设上(渠道按量计费时会用到) | |||
| require.NoError(t, ratio_setting.UpdateModelRatioByJSONString(`{"`+testModelUsePriceSwitch+`":15}`)) | |||
| require.NoError(t, ratio_setting.UpdateCompletionRatioByJSONString(`{"`+testModelUsePriceSwitch+`":3}`)) | |||
| // 全局扩展比率 | |||
| require.NoError(t, ratio_setting.UpdateCacheRatioByJSONString(`{"`+testModelUsePriceSwitch+`":0.1}`)) | |||
| require.NoError(t, ratio_setting.UpdateCreateCacheRatioByJSONString(`{"`+testModelUsePriceSwitch+`":1.25}`)) | |||
| require.NoError(t, ratio_setting.UpdateImageRatioByJSONString(`{"`+testModelUsePriceSwitch+`":2.0}`)) | |||
| require.NoError(t, ratio_setting.UpdateAudioRatioByJSONString(`{"`+testModelUsePriceSwitch+`":5.0}`)) | |||
| require.NoError(t, ratio_setting.UpdateAudioCompletionRatioByJSONString(`{"`+testModelUsePriceSwitch+`":3.0}`)) | |||
| // 渠道定价:按量计费(QuotaTypeByTokens),扩展比率全部为 0(用全局) | |||
| cp := &model.ChannelPricing{ | |||
| ModelName: testModelUsePriceSwitch, | |||
| ChannelId: 9901, | |||
| QuotaType: model.QuotaTypeByTokens, | |||
| ModelRatio: 15, | |||
| CompletionRatio: 3, | |||
| // 扩展比率全部为 0 → 意味着回退全局 | |||
| } | |||
| require.NoError(t, cp.Insert()) | |||
| t.Cleanup(func() { | |||
| model.DB = origDB | |||
| sqlDB.Close() | |||
| // 清理全局定价 | |||
| ratio_setting.UpdateModelPriceByJSONString(`{}`) | |||
| ratio_setting.UpdateModelRatioByJSONString(`{}`) | |||
| ratio_setting.UpdateCompletionRatioByJSONString(`{}`) | |||
| ratio_setting.UpdateCacheRatioByJSONString(`{}`) | |||
| ratio_setting.UpdateCreateCacheRatioByJSONString(`{}`) | |||
| ratio_setting.UpdateImageRatioByJSONString(`{}`) | |||
| ratio_setting.UpdateAudioRatioByJSONString(`{}`) | |||
| ratio_setting.UpdateAudioCompletionRatioByJSONString(`{}`) | |||
| common.UsingSQLite = origUsingSQLite | |||
| common.RedisEnabled = origRedisEnabled | |||
| require.NoError(t, sqlDB.Close()) | |||
| require.NoError(t, ratio_setting.UpdateModelPriceByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateModelRatioByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateCompletionRatioByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateCacheRatioByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateCreateCacheRatioByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateImageRatioByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateAudioRatioByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateAudioCompletionRatioByJSONString(`{}`)) | |||
| }) | |||
| } | |||
| @@ -79,88 +68,240 @@ func buildTestContext(t *testing.T) *gin.Context { | |||
| gin.SetMode(gin.TestMode) | |||
| w := httptest.NewRecorder() | |||
| c, _ := gin.CreateTestContext(w) | |||
| c.Request, _ = http.NewRequest("POST", "/v1/chat/completions", nil) | |||
| req, err := http.NewRequest(http.MethodPost, "/v1/chat/completions", nil) | |||
| require.NoError(t, err) | |||
| c.Request = req | |||
| return c | |||
| } | |||
| // TestGlobalRatiosFallbackWhenGlobalUsePriceTrue 验证核心场景: | |||
| // 全局按次计费 → 渠道按量计费 → PriceData 中扩展比率仍应被全局默认值填充 | |||
| // 而不是保持 0(否则后续 UpdatePriceDataForChannelPricing 切换为按量时会丢失比率) | |||
| func TestGlobalRatiosFallbackWhenGlobalUsePriceTrue(t *testing.T) { | |||
| func TestGlobalRatiosFilledWhenUsePriceTrue(t *testing.T) { | |||
| setupUsePriceSwitchTest(t) | |||
| c := buildTestContext(t) | |||
| // 模拟渠道选择后的场景:ChannelMeta 不为空,能查到渠道定价 | |||
| info := &relaycommon.RelayInfo{ | |||
| OriginModelName: testModelUsePriceSwitch, | |||
| ChannelMeta: &relaycommon.ChannelMeta{ | |||
| ChannelId: 9901, | |||
| }, | |||
| UserSetting: dto.UserSetting{}, | |||
| UserSetting: dto.UserSetting{}, | |||
| } | |||
| meta := &types.TokenCountMeta{} | |||
| priceData, err := ModelPriceHelper(c, info, 100, meta) | |||
| priceData, err := ModelPriceHelper(c, info, 100, &types.TokenCountMeta{}) | |||
| require.NoError(t, err) | |||
| // 断言:渠道按量计费覆盖了全局按次,扩展比率必须被全局默认值填充,不能是 0 | |||
| assert.Equal(t, 0.1, priceData.CacheRatio, "CacheRatio 应从全局回退填充") | |||
| assert.Equal(t, 1.25, priceData.CacheCreationRatio, "CacheCreationRatio 应从全局回退填充") | |||
| assert.Equal(t, 2.0, priceData.ImageRatio, "ImageRatio 应从全局回退填充") | |||
| assert.Equal(t, 5.0, priceData.AudioRatio, "AudioRatio 应从全局回退填充") | |||
| assert.Equal(t, 3.0, priceData.AudioCompletionRatio, "AudioCompletionRatio 应从全局回退填充") | |||
| assert.True(t, priceData.UsePrice) | |||
| assert.Equal(t, 0.5, priceData.ModelPrice) | |||
| assert.Equal(t, 0.1, priceData.CacheRatio) | |||
| assert.Equal(t, 1.25, priceData.CacheCreationRatio) | |||
| assert.Equal(t, 2.0, priceData.ImageRatio) | |||
| assert.Equal(t, 5.0, priceData.AudioRatio) | |||
| assert.Equal(t, 3.0, priceData.AudioCompletionRatio) | |||
| } | |||
| func TestGlobalRatiosFilledWhenUseRatioMode(t *testing.T) { | |||
| setupUsePriceSwitchTest(t) | |||
| c := buildTestContext(t) | |||
| require.NoError(t, ratio_setting.UpdateModelPriceByJSONString(`{}`)) | |||
| info := &relaycommon.RelayInfo{ | |||
| OriginModelName: testModelUsePriceSwitch, | |||
| UserSetting: dto.UserSetting{}, | |||
| } | |||
| priceData, err := ModelPriceHelper(c, info, 100, &types.TokenCountMeta{}) | |||
| require.NoError(t, err) | |||
| assert.False(t, priceData.UsePrice) | |||
| assert.Equal(t, 15.0, priceData.ModelRatio) | |||
| assert.Equal(t, 3.0, priceData.CompletionRatio) | |||
| assert.Equal(t, 0.1, priceData.CacheRatio) | |||
| assert.Equal(t, 2.0, priceData.ImageRatio) | |||
| } | |||
| func TestHandleGroupRatio_DefaultGroup(t *testing.T) { | |||
| setupUsePriceSwitchTest(t) | |||
| c := buildTestContext(t) | |||
| info := &relaycommon.RelayInfo{ | |||
| UsingGroup: "default", | |||
| UserGroup: "default", | |||
| } | |||
| groupRatioInfo := HandleGroupRatio(c, info) | |||
| assert.Equal(t, 1.0, groupRatioInfo.GroupRatio) | |||
| assert.False(t, groupRatioInfo.HasSpecialRatio) | |||
| assert.Equal(t, -1.0, groupRatioInfo.GroupSpecialRatio) | |||
| } | |||
| func TestHandleGroupRatio_CustomGroupRatio(t *testing.T) { | |||
| setupUsePriceSwitchTest(t) | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1,"premium":0.7}`)) | |||
| c := buildTestContext(t) | |||
| info := &relaycommon.RelayInfo{ | |||
| UsingGroup: "premium", | |||
| UserGroup: "default", | |||
| } | |||
| groupRatioInfo := HandleGroupRatio(c, info) | |||
| assert.Equal(t, 0.7, groupRatioInfo.GroupRatio) | |||
| assert.False(t, groupRatioInfo.HasSpecialRatio) | |||
| // cleanup | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1}`)) | |||
| } | |||
| func TestHandleGroupRatio_SpecialGroupGroupRatio(t *testing.T) { | |||
| setupUsePriceSwitchTest(t) | |||
| require.NoError(t, ratio_setting.UpdateGroupGroupRatioByJSONString(`{"vip":{"premium":0.8}}`)) | |||
| c := buildTestContext(t) | |||
| info := &relaycommon.RelayInfo{ | |||
| UsingGroup: "premium", | |||
| UserGroup: "vip", | |||
| } | |||
| groupRatioInfo := HandleGroupRatio(c, info) | |||
| assert.Equal(t, 0.8, groupRatioInfo.GroupRatio) | |||
| assert.Equal(t, 0.8, groupRatioInfo.GroupSpecialRatio) | |||
| assert.True(t, groupRatioInfo.HasSpecialRatio) | |||
| // cleanup | |||
| require.NoError(t, ratio_setting.UpdateGroupGroupRatioByJSONString(`{}`)) | |||
| } | |||
| func TestHandleGroupRatio_AutoGroupOverridesUsingGroup(t *testing.T) { | |||
| setupUsePriceSwitchTest(t) | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1,"vip-channel":0.6}`)) | |||
| c := buildTestContext(t) | |||
| // Simulate: distributor set auto_group = "vip-channel" in context | |||
| c.Set("auto_group", "vip-channel") | |||
| info := &relaycommon.RelayInfo{ | |||
| UsingGroup: "default", // will be overridden | |||
| UserGroup: "default", | |||
| } | |||
| groupRatioInfo := HandleGroupRatio(c, info) | |||
| // UsingGroup should be updated to "vip-channel" | |||
| assert.Equal(t, "vip-channel", info.UsingGroup) | |||
| // GroupRatio should reflect vip-channel's ratio | |||
| assert.Equal(t, 0.6, groupRatioInfo.GroupRatio) | |||
| // cleanup | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1}`)) | |||
| } | |||
| // TestGlobalRatiosFallbackWithoutChannelPricing 验证无渠道定价时全局比率也正常回退 | |||
| func TestGlobalRatiosFallbackWithoutChannelPricing(t *testing.T) { | |||
| func TestHandleGroupRatio_AutoGroupWithSpecialRatio(t *testing.T) { | |||
| setupUsePriceSwitchTest(t) | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1,"premium":1.0}`)) | |||
| require.NoError(t, ratio_setting.UpdateGroupGroupRatioByJSONString(`{"vip":{"premium":0.5}}`)) | |||
| c := buildTestContext(t) | |||
| c.Set("auto_group", "premium") | |||
| info := &relaycommon.RelayInfo{ | |||
| UsingGroup: "default", | |||
| UserGroup: "vip", | |||
| } | |||
| groupRatioInfo := HandleGroupRatio(c, info) | |||
| assert.Equal(t, "premium", info.UsingGroup) | |||
| // Should use the special ratio (0.5) from vip->premium, not the generic group ratio (1.0) | |||
| assert.Equal(t, 0.5, groupRatioInfo.GroupRatio) | |||
| assert.True(t, groupRatioInfo.HasSpecialRatio) | |||
| // cleanup | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1}`)) | |||
| require.NoError(t, ratio_setting.UpdateGroupGroupRatioByJSONString(`{}`)) | |||
| } | |||
| // --- ModelPriceHelper with group ratio --- | |||
| func TestModelPriceHelper_GroupRatioAffectsPreConsume(t *testing.T) { | |||
| setupUsePriceSwitchTest(t) | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1,"discount":0.5}`)) | |||
| c := buildTestContext(t) | |||
| // 无 ChannelMeta → 渠道定价不可用,走全局 | |||
| info := &relaycommon.RelayInfo{ | |||
| OriginModelName: testModelUsePriceSwitch, | |||
| UsingGroup: "discount", | |||
| UserGroup: "default", | |||
| UserSetting: dto.UserSetting{}, | |||
| } | |||
| meta := &types.TokenCountMeta{} | |||
| priceData, err := ModelPriceHelper(c, info, 100, meta) | |||
| priceData, err := ModelPriceHelper(c, info, 100, &types.TokenCountMeta{}) | |||
| require.NoError(t, err) | |||
| // 全局是按次计费,UsePrice=true,但比率仍应被填充 | |||
| // model price 0.5, group ratio 0.5 => preConsume = 0.5 * QuotaPerUnit * 0.5 | |||
| assert.Equal(t, 0.5, priceData.GroupRatioInfo.GroupRatio) | |||
| assert.True(t, priceData.UsePrice) | |||
| assert.Equal(t, 0.1, priceData.CacheRatio, "CacheRatio 应从全局回退填充") | |||
| assert.Equal(t, 2.0, priceData.ImageRatio, "ImageRatio 应从全局回退填充") | |||
| // 0.5 (modelPrice) * 500000 (QuotaPerUnit) * 0.5 (groupRatio) = 125000 | |||
| expectedQuota := int(0.5 * float64(common.QuotaPerUnit) * 0.5) | |||
| assert.Equal(t, expectedQuota, priceData.QuotaToPreConsume) | |||
| // cleanup | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1}`)) | |||
| } | |||
| // TestChannelRatiosOverrideGlobal 验证渠道非零值优先于全局 | |||
| func TestChannelRatiosOverrideGlobal(t *testing.T) { | |||
| func TestModelPriceHelper_RatioModeWithGroupRatio(t *testing.T) { | |||
| setupUsePriceSwitchTest(t) | |||
| // Remove model price so it falls back to ratio mode | |||
| require.NoError(t, ratio_setting.UpdateModelPriceByJSONString(`{}`)) | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1,"double":2.0}`)) | |||
| c := buildTestContext(t) | |||
| // 渠道定价设了自定义 cacheRatio=0.5 | |||
| cp := &model.ChannelPricing{ | |||
| ModelName: testModelUsePriceSwitch, | |||
| ChannelId: 9902, | |||
| QuotaType: model.QuotaTypeByTokens, | |||
| ModelRatio: 15, | |||
| CompletionRatio: 3, | |||
| CacheRatio: 0.5, // 渠道自定义,覆盖全局 0.1 | |||
| // 其他扩展比率保持 0,用全局 | |||
| info := &relaycommon.RelayInfo{ | |||
| OriginModelName: testModelUsePriceSwitch, | |||
| UsingGroup: "double", | |||
| UserGroup: "default", | |||
| UserSetting: dto.UserSetting{}, | |||
| } | |||
| require.NoError(t, cp.Insert()) | |||
| priceData, err := ModelPriceHelper(c, info, 100, &types.TokenCountMeta{}) | |||
| require.NoError(t, err) | |||
| assert.False(t, priceData.UsePrice) | |||
| assert.Equal(t, 15.0, priceData.ModelRatio) | |||
| assert.Equal(t, 2.0, priceData.GroupRatioInfo.GroupRatio) | |||
| // ratio = modelRatio(15) * groupRatio(2.0) = 30 | |||
| // preConsumedQuota = int(preConsumedTokens * 30) | |||
| expectedRatio := 15.0 * 2.0 | |||
| assert.Equal(t, expectedRatio, priceData.ModelRatio*priceData.GroupRatioInfo.GroupRatio) | |||
| // cleanup | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1}`)) | |||
| } | |||
| func TestModelPriceHelper_FreeGroup(t *testing.T) { | |||
| setupUsePriceSwitchTest(t) | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1,"free":0}`)) | |||
| // Disable pre-consume for free models so freeModel flag is set | |||
| origSetting := operation_setting.GetQuotaSetting() | |||
| operation_setting.GetQuotaSetting().EnableFreeModelPreConsume = false | |||
| t.Cleanup(func() { | |||
| operation_setting.GetQuotaSetting().EnableFreeModelPreConsume = origSetting.EnableFreeModelPreConsume | |||
| }) | |||
| c := buildTestContext(t) | |||
| info := &relaycommon.RelayInfo{ | |||
| OriginModelName: testModelUsePriceSwitch, | |||
| ChannelMeta: &relaycommon.ChannelMeta{ | |||
| ChannelId: 9902, | |||
| }, | |||
| UserSetting: dto.UserSetting{}, | |||
| UsingGroup: "free", | |||
| UserGroup: "default", | |||
| UserSetting: dto.UserSetting{}, | |||
| } | |||
| priceData, err := ModelPriceHelper(c, info, 100, &types.TokenCountMeta{}) | |||
| require.NoError(t, err) | |||
| assert.Equal(t, 0.5, priceData.CacheRatio, "渠道自定义 CacheRatio 应覆盖全局") | |||
| assert.Equal(t, 2.0, priceData.ImageRatio, "ImageRatio 未设渠道值,应从全局回退") | |||
| assert.Equal(t, 5.0, priceData.AudioRatio, "AudioRatio 未设渠道值,应从全局回退") | |||
| // group ratio = 0 means free model | |||
| assert.Equal(t, 0.0, priceData.GroupRatioInfo.GroupRatio) | |||
| assert.True(t, priceData.FreeModel) | |||
| assert.Equal(t, 0, priceData.QuotaToPreConsume) | |||
| // cleanup | |||
| require.NoError(t, ratio_setting.UpdateGroupRatioByJSONString(`{"default":1,"vip":1,"svip":1}`)) | |||
| } | |||
| @@ -33,7 +33,6 @@ func SetApiRouter(router *gin.Engine) { | |||
| apiRouter.GET("/home_page_content", controller.GetHomePageContent) | |||
| apiRouter.GET("/pricing", middleware.TryUserAuth(), controller.GetPricing) | |||
| apiRouter.GET("/pricing/user/*model", middleware.TryUserAuth(), controller.GetUserPricing) | |||
| apiRouter.GET("/channel-pricing/model/*name", middleware.TryUserAuth(), controller.GetChannelPricingByModelWithChannelInfo) | |||
| apiRouter.GET("/captcha", controller.GetCaptcha) | |||
| apiRouter.GET("/verification", middleware.EmailVerificationRateLimit(), middleware.TurnstileCheck(), controller.SendEmailVerification) | |||
| apiRouter.GET("/reset_password", middleware.CriticalRateLimit(), middleware.TurnstileCheck(), controller.SendPasswordResetEmail) | |||
| @@ -59,6 +58,7 @@ func SetApiRouter(router *gin.Engine) { | |||
| apiRouter.POST("/verify", middleware.UserAuth(), middleware.CriticalRateLimit(), controller.UniversalVerify) | |||
| userRoute := apiRouter.Group("/user") | |||
| userRoute.Use(middleware.CORS()) | |||
| { | |||
| userRoute.POST("/register", middleware.CriticalRateLimit(), middleware.TurnstileCheck(), controller.Register) | |||
| userRoute.POST("/login", middleware.CriticalRateLimit(), middleware.TurnstileCheck(), controller.Login) | |||
| @@ -77,8 +77,6 @@ func SetApiRouter(router *gin.Engine) { | |||
| selfRoute.GET("/self/groups", controller.GetUserGroups) | |||
| selfRoute.GET("/self", controller.GetSelf) | |||
| selfRoute.GET("/models", controller.GetUserModels) | |||
| selfRoute.GET("/model_channels", controller.GetModelChannels) | |||
| selfRoute.GET("/channels", controller.GetUserChannelsForBinding) | |||
| selfRoute.PUT("/self", controller.UpdateSelf) | |||
| selfRoute.DELETE("/self", controller.DeleteSelf) | |||
| selfRoute.GET("/token", controller.GenerateAccessToken) | |||
| @@ -193,41 +191,6 @@ func SetApiRouter(router *gin.Engine) { | |||
| optionRoute.POST("/migrate_console_setting", controller.MigrateConsoleSetting) // 用于迁移检测的旧键,下个版本会删除 | |||
| } | |||
| // 渠道定价路由(管理员权限) | |||
| channelPricingRoute := apiRouter.Group("/channel_pricing") | |||
| channelPricingRoute.Use(middleware.AdminAuth()) | |||
| { | |||
| channelPricingRoute.GET("/", controller.GetAllChannelPricing) | |||
| channelPricingRoute.GET("/with_tags", controller.GetChannelPricingWithTags) | |||
| channelPricingRoute.GET("/model/:name", controller.GetChannelPricingByModel) | |||
| channelPricingRoute.POST("/", controller.CreateChannelPricing) | |||
| channelPricingRoute.POST("/batch", controller.BatchCreateChannelPricing) | |||
| channelPricingRoute.POST("/copy_global/:channel_id", controller.CopyGlobalPricing) | |||
| channelPricingRoute.DELETE("/:id", controller.DeleteChannelPricing) | |||
| channelPricingRoute.POST("/set_default", controller.SetDefaultChannel) | |||
| channelPricingRoute.DELETE("/default/*name", controller.ClearDefaultChannel) | |||
| } | |||
| // 用户倍率路由(管理员权限) | |||
| userChannelRatioRoute := apiRouter.Group("/user_channel_ratio") | |||
| userChannelRatioRoute.Use(middleware.AdminAuth()) | |||
| { | |||
| userChannelRatioRoute.GET("/:user_id", controller.GetUserChannelRatios) | |||
| userChannelRatioRoute.POST("/", controller.CreateUserChannelRatio) | |||
| userChannelRatioRoute.PUT("/:id", controller.UpdateUserChannelRatio) | |||
| userChannelRatioRoute.DELETE("/:id", controller.DeleteUserChannelRatio) | |||
| } | |||
| // 定价标签路由(管理员权限) | |||
| pricingTagRoute := apiRouter.Group("/pricing_tag") | |||
| pricingTagRoute.Use(middleware.AdminAuth()) | |||
| { | |||
| pricingTagRoute.GET("/", controller.GetAllPricingTags) | |||
| pricingTagRoute.POST("/", controller.CreatePricingTag) | |||
| pricingTagRoute.PUT("/:id", controller.UpdatePricingTag) | |||
| pricingTagRoute.DELETE("/:id", controller.DeletePricingTag) | |||
| } | |||
| // 邮箱后缀额度规则路由(管理员权限) | |||
| emailQuotaRuleRoute := apiRouter.Group("/email_quota_rule") | |||
| emailQuotaRuleRoute.Use(middleware.AdminAuth()) | |||
| @@ -442,5 +405,30 @@ func SetApiRouter(router *gin.Engine) { | |||
| syncRoute.POST("/quota/batch-deduct", controller.BatchDeductQuota) | |||
| syncRoute.GET("/config", controller.GetSyncConfig) | |||
| } | |||
| migrationRoute := apiRouter.Group("/internal/migration") | |||
| migrationRoute.Use(middleware.SyncAuth()) | |||
| { | |||
| migrationRoute.GET("/users/check-synced-copy", controller.CheckSyncedCopy) | |||
| migrationRoute.GET("/users", controller.ListMigrationUsers) | |||
| migrationRoute.POST("/users/query", controller.QueryMigrationUsers) | |||
| migrationRoute.GET("/users/:id", controller.GetMigrationUser) | |||
| migrationRoute.POST("/users/:id/convert-to-synced", controller.ConvertMigrationUserToSynced) | |||
| } | |||
| userMigrationRoute := apiRouter.Group("/user-migrations") | |||
| userMigrationRoute.Use(middleware.RootAuth(), middleware.MasterNodeOnly()) | |||
| { | |||
| userMigrationRoute.GET("/candidate-users", controller.ListMigrationCandidateUsers) | |||
| userMigrationRoute.POST("/batches", controller.CreateUserMigrationBatch) | |||
| userMigrationRoute.GET("/batches", controller.ListUserMigrationBatches) | |||
| userMigrationRoute.GET("/batches/:id", controller.GetUserMigrationBatch) | |||
| userMigrationRoute.POST("/batches/:id/scan", controller.ScanUserMigrationBatch) | |||
| userMigrationRoute.POST("/batches/:id/execute", controller.ExecuteUserMigrationBatch) | |||
| userMigrationRoute.POST("/batches/:id/retry", controller.RetryUserMigrationBatch) | |||
| userMigrationRoute.POST("/batches/:id/cancel", controller.CancelUserMigrationBatch) | |||
| userMigrationRoute.POST("/batches/:id/verify", controller.VerifyUserMigrationBatch) | |||
| userMigrationRoute.POST("/items/:id/resolve", controller.ResolveUserMigrationItem) | |||
| } | |||
| } | |||
| } | |||
| @@ -1,6 +1,8 @@ | |||
| package router | |||
| import ( | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/common/metrics" | |||
| "github.com/QuantumNous/new-api/constant" | |||
| "github.com/QuantumNous/new-api/controller" | |||
| "github.com/QuantumNous/new-api/middleware" | |||
| @@ -10,7 +12,19 @@ import ( | |||
| "github.com/gin-gonic/gin" | |||
| ) | |||
| // relayHandler returns a handler that calls controller.Relay with the given format. | |||
| func relayHandler(format types.RelayFormat) gin.HandlerFunc { | |||
| return func(c *gin.Context) { | |||
| controller.Relay(c, format) | |||
| } | |||
| } | |||
| func SetRelayRouter(router *gin.Engine) { | |||
| if metrics.IsEnabled() { | |||
| metricsPath := common.GetEnvOrDefaultString("METRICS_PATH", "/metrics") | |||
| router.GET(metricsPath, gin.WrapH(metrics.NewMetricsHandler())) | |||
| } | |||
| router.Use(middleware.CORS()) | |||
| router.Use(middleware.DecompressRequestMiddleware()) | |||
| router.Use(middleware.BodyStorageCleanup()) // 清理请求体存储 | |||
| @@ -65,87 +79,58 @@ func SetRelayRouter(router *gin.Engine) { | |||
| relayV1Router := router.Group("/v1") | |||
| relayV1Router.Use(middleware.SystemPerformanceCheck()) | |||
| relayV1Router.Use(middleware.TokenAuth()) | |||
| if metrics.IsEnabled() { | |||
| relayV1Router.Use(metrics.MetricsMiddleware(metrics.GetMetrics())) | |||
| } | |||
| relayV1Router.Use(middleware.ModelRequestRateLimit()) | |||
| relayV1Router.Use(middleware.RelayCaptureMiddleware()) | |||
| { | |||
| // WebSocket 路由(统一到 Relay) | |||
| wsRouter := relayV1Router.Group("") | |||
| wsRouter.Use(middleware.Distribute()) | |||
| wsRouter.GET("/realtime", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAIRealtime) | |||
| }) | |||
| wsRouter.Use(middleware.EndpointFormatGuard()) | |||
| wsRouter.GET("/realtime", relayHandler(types.RelayFormatOpenAIRealtime)) | |||
| } | |||
| { | |||
| //http router | |||
| httpRouter := relayV1Router.Group("") | |||
| httpRouter.Use(middleware.Distribute()) | |||
| httpRouter.Use(middleware.EndpointFormatGuard()) | |||
| httpRouter.Use(middleware.UserModelRateLimit()) | |||
| // claude related routes | |||
| httpRouter.POST("/messages", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatClaude) | |||
| }) | |||
| httpRouter.POST("/messages", relayHandler(types.RelayFormatClaude)) | |||
| // chat related routes | |||
| httpRouter.POST("/completions", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAI) | |||
| }) | |||
| httpRouter.POST("/chat/completions", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAI) | |||
| }) | |||
| httpRouter.POST("/completions", relayHandler(types.RelayFormatOpenAI)) | |||
| httpRouter.POST("/chat/completions", relayHandler(types.RelayFormatOpenAI)) | |||
| // response related routes | |||
| httpRouter.POST("/responses", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAIResponses) | |||
| }) | |||
| httpRouter.POST("/responses/compact", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAIResponsesCompaction) | |||
| }) | |||
| httpRouter.POST("/responses", relayHandler(types.RelayFormatOpenAIResponses)) | |||
| httpRouter.POST("/responses/compact", relayHandler(types.RelayFormatOpenAIResponsesCompaction)) | |||
| // image related routes | |||
| httpRouter.POST("/edits", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAIImage) | |||
| }) | |||
| httpRouter.POST("/images/generations", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAIImage) | |||
| }) | |||
| httpRouter.POST("/images/edits", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAIImage) | |||
| }) | |||
| httpRouter.POST("/edits", relayHandler(types.RelayFormatOpenAIImage)) | |||
| httpRouter.POST("/images/generations", relayHandler(types.RelayFormatOpenAIImage)) | |||
| httpRouter.POST("/images/edits", relayHandler(types.RelayFormatOpenAIImage)) | |||
| // embedding related routes | |||
| httpRouter.POST("/embeddings", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatEmbedding) | |||
| }) | |||
| httpRouter.POST("/embeddings", relayHandler(types.RelayFormatEmbedding)) | |||
| // audio related routes | |||
| httpRouter.POST("/audio/transcriptions", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAIAudio) | |||
| }) | |||
| httpRouter.POST("/audio/translations", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAIAudio) | |||
| }) | |||
| httpRouter.POST("/audio/speech", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAIAudio) | |||
| }) | |||
| httpRouter.POST("/audio/transcriptions", relayHandler(types.RelayFormatOpenAIAudio)) | |||
| httpRouter.POST("/audio/translations", relayHandler(types.RelayFormatOpenAIAudio)) | |||
| httpRouter.POST("/audio/speech", relayHandler(types.RelayFormatOpenAIAudio)) | |||
| // rerank related routes | |||
| httpRouter.POST("/rerank", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatRerank) | |||
| }) | |||
| httpRouter.POST("/rerank", relayHandler(types.RelayFormatRerank)) | |||
| // gemini relay routes | |||
| httpRouter.POST("/engines/:model/embeddings", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatGemini) | |||
| }) | |||
| httpRouter.POST("/models/*path", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatGemini) | |||
| }) | |||
| httpRouter.POST("/engines/:model/embeddings", relayHandler(types.RelayFormatGemini)) | |||
| httpRouter.POST("/models/*path", relayHandler(types.RelayFormatGemini)) | |||
| // other relay routes | |||
| httpRouter.POST("/moderations", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatOpenAI) | |||
| }) | |||
| httpRouter.POST("/moderations", relayHandler(types.RelayFormatOpenAI)) | |||
| // not implemented | |||
| httpRouter.POST("/images/variations", controller.RelayNotImplemented) | |||
| @@ -183,15 +168,17 @@ func SetRelayRouter(router *gin.Engine) { | |||
| relayGeminiRouter := router.Group("/v1beta") | |||
| relayGeminiRouter.Use(middleware.SystemPerformanceCheck()) | |||
| relayGeminiRouter.Use(middleware.TokenAuth()) | |||
| if metrics.IsEnabled() { | |||
| relayGeminiRouter.Use(metrics.MetricsMiddleware(metrics.GetMetrics())) | |||
| } | |||
| relayGeminiRouter.Use(middleware.ModelRequestRateLimit()) | |||
| relayGeminiRouter.Use(middleware.RelayCaptureMiddleware()) | |||
| relayGeminiRouter.Use(middleware.Distribute()) | |||
| relayGeminiRouter.Use(middleware.EndpointFormatGuard()) | |||
| relayGeminiRouter.Use(middleware.UserModelRateLimit()) | |||
| { | |||
| // Gemini API 路径格式: /v1beta/models/{model_name}:{action} | |||
| relayGeminiRouter.POST("/models/*path", func(c *gin.Context) { | |||
| controller.Relay(c, types.RelayFormatGemini) | |||
| }) | |||
| relayGeminiRouter.POST("/models/*path", relayHandler(types.RelayFormatGemini)) | |||
| } | |||
| } | |||
| @@ -0,0 +1,194 @@ | |||
| package router | |||
| import ( | |||
| "net/http" | |||
| "net/http/httptest" | |||
| "testing" | |||
| "github.com/QuantumNous/new-api/common" | |||
| "github.com/QuantumNous/new-api/controller" | |||
| "github.com/QuantumNous/new-api/middleware" | |||
| "github.com/QuantumNous/new-api/model" | |||
| "github.com/gin-contrib/sessions" | |||
| "github.com/gin-contrib/sessions/cookie" | |||
| "github.com/gin-gonic/gin" | |||
| "github.com/glebarez/sqlite" | |||
| "github.com/stretchr/testify/require" | |||
| "gorm.io/gorm" | |||
| ) | |||
| func setupUserMigrationRouterDB(t *testing.T) *gorm.DB { | |||
| t.Helper() | |||
| db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{}) | |||
| require.NoError(t, err) | |||
| sqlDB, err := db.DB() | |||
| require.NoError(t, err) | |||
| sqlDB.SetMaxOpenConns(1) | |||
| require.NoError(t, db.AutoMigrate( | |||
| &model.User{}, | |||
| &model.CustomOAuthProvider{}, | |||
| &model.UserOAuthBinding{}, | |||
| &model.UserMigrationBatch{}, | |||
| &model.UserMigrationItem{}, | |||
| &model.MigrationQuotaGrant{}, | |||
| )) | |||
| return db | |||
| } | |||
| func setupUserMigrationRootRouter() *gin.Engine { | |||
| gin.SetMode(gin.TestMode) | |||
| r := gin.New() | |||
| store := cookie.NewStore([]byte("test-session-secret")) | |||
| store.Options(sessions.Options{ | |||
| Path: "/", | |||
| MaxAge: 3600, | |||
| HttpOnly: true, | |||
| Secure: false, | |||
| }) | |||
| r.Use(sessions.Sessions("session", store)) | |||
| userMigrationRoute := r.Group("/api/user-migrations") | |||
| userMigrationRoute.Use(func(c *gin.Context) { | |||
| session := sessions.Default(c) | |||
| session.Set("username", "root") | |||
| session.Set("role", common.RoleRootUser) | |||
| session.Set("id", 1) | |||
| session.Set("status", common.UserStatusEnabled) | |||
| session.Save() | |||
| c.Request.Header.Set("New-Api-User", "1") | |||
| c.Next() | |||
| }, middleware.RootAuth(), middleware.MasterNodeOnly()) | |||
| { | |||
| userMigrationRoute.GET("/batches", controller.ListUserMigrationBatches) | |||
| userMigrationRoute.POST("/batches/:id/execute", controller.ExecuteUserMigrationBatch) | |||
| userMigrationRoute.POST("/batches/:id/cancel", controller.CancelUserMigrationBatch) | |||
| } | |||
| return r | |||
| } | |||
| func setupUserMigrationAdminRouter() *gin.Engine { | |||
| gin.SetMode(gin.TestMode) | |||
| r := gin.New() | |||
| store := cookie.NewStore([]byte("test-session-secret")) | |||
| store.Options(sessions.Options{ | |||
| Path: "/", | |||
| MaxAge: 3600, | |||
| HttpOnly: true, | |||
| Secure: false, | |||
| }) | |||
| r.Use(sessions.Sessions("session", store)) | |||
| userMigrationRoute := r.Group("/api/user-migrations") | |||
| userMigrationRoute.Use(func(c *gin.Context) { | |||
| session := sessions.Default(c) | |||
| session.Set("username", "admin") | |||
| session.Set("role", common.RoleAdminUser) | |||
| session.Set("id", 2) | |||
| session.Set("status", common.UserStatusEnabled) | |||
| session.Save() | |||
| c.Request.Header.Set("New-Api-User", "2") | |||
| c.Next() | |||
| }, middleware.RootAuth(), middleware.MasterNodeOnly()) | |||
| { | |||
| userMigrationRoute.GET("/batches", controller.ListUserMigrationBatches) | |||
| userMigrationRoute.POST("/batches/:id/execute", controller.ExecuteUserMigrationBatch) | |||
| userMigrationRoute.POST("/batches/:id/cancel", controller.CancelUserMigrationBatch) | |||
| } | |||
| return r | |||
| } | |||
| func TestUserMigrationRoutes_RootCanAccess(t *testing.T) { | |||
| db := setupUserMigrationRouterDB(t) | |||
| orig := model.DB | |||
| origMaster := common.IsMasterNode | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| common.IsMasterNode = origMaster | |||
| }() | |||
| common.IsMasterNode = true | |||
| require.NoError(t, db.Create(&model.UserMigrationBatch{ | |||
| Id: 1, | |||
| Name: "router-batch", | |||
| SourceRegion: "ov", | |||
| TargetRegion: "cn", | |||
| OperatorId: 1, | |||
| Status: model.UserMigrationBatchStatusDraft, | |||
| }).Error) | |||
| router := setupUserMigrationRootRouter() | |||
| req := httptest.NewRequest(http.MethodGet, "/api/user-migrations/batches", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| req = httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/execute", nil) | |||
| w = httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.NotEqual(t, http.StatusUnauthorized, w.Code) | |||
| req = httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/cancel", nil) | |||
| w = httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.NotEqual(t, http.StatusUnauthorized, w.Code) | |||
| } | |||
| func TestUserMigrationRoutes_AdminForbidden(t *testing.T) { | |||
| db := setupUserMigrationRouterDB(t) | |||
| orig := model.DB | |||
| origMaster := common.IsMasterNode | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| common.IsMasterNode = origMaster | |||
| }() | |||
| common.IsMasterNode = true | |||
| router := setupUserMigrationAdminRouter() | |||
| req := httptest.NewRequest(http.MethodGet, "/api/user-migrations/batches", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| require.Contains(t, w.Body.String(), "success\":false") | |||
| req = httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/execute", nil) | |||
| w = httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| require.Contains(t, w.Body.String(), "success\":false") | |||
| req = httptest.NewRequest(http.MethodPost, "/api/user-migrations/batches/1/cancel", nil) | |||
| w = httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusOK, w.Code) | |||
| require.Contains(t, w.Body.String(), "success\":false") | |||
| } | |||
| func TestUserMigrationRoutes_RootForbiddenOnSlaveNode(t *testing.T) { | |||
| db := setupUserMigrationRouterDB(t) | |||
| orig := model.DB | |||
| origMaster := common.IsMasterNode | |||
| model.DB = db | |||
| defer func() { | |||
| model.DB = orig | |||
| common.IsMasterNode = origMaster | |||
| }() | |||
| common.IsMasterNode = false | |||
| router := setupUserMigrationRootRouter() | |||
| req := httptest.NewRequest(http.MethodGet, "/api/user-migrations/batches", nil) | |||
| w := httptest.NewRecorder() | |||
| router.ServeHTTP(w, req) | |||
| require.Equal(t, http.StatusForbidden, w.Code) | |||
| require.Contains(t, w.Body.String(), "only master node") | |||
| } | |||
| @@ -10,15 +10,12 @@ | |||
| set -euo pipefail | |||
| # ==================== 配置 ==================== | |||
| DOCKER_REGISTRY="registry.cn-hangzhou.aliyuncs.com/fengsilin/new-api" | |||
| PROJECT_DIR="/mnt/d/code/new-api" | |||
| # 测试节点 | |||
| CN_DIR="/root/new-api-cn" | |||
| OV_DIR="/root/new-api-ov" | |||
| # ==================== 颜色 ==================== | |||
| RED='\033[0;31m' | |||
| GREEN='\033[0;32m' | |||
| YELLOW='\033[1;33m' | |||
| @@ -38,14 +35,13 @@ section() { | |||
| echo -e " ${BOLD}=========================================${NC}" | |||
| } | |||
| # ==================== 参数解析 ==================== | |||
| TARGET="all" | |||
| NO_BUILD=false | |||
| for arg in "$@"; do | |||
| case "$arg" in | |||
| cn) TARGET="cn" ;; | |||
| ov) TARGET="ov" ;; | |||
| cn) TARGET="cn" ;; | |||
| ov) TARGET="ov" ;; | |||
| --no-build) NO_BUILD=true ;; | |||
| -h|--help) | |||
| echo "用法: $0 [cn|ov] [--no-build]" | |||
| @@ -62,14 +58,16 @@ for arg in "$@"; do | |||
| esac | |||
| done | |||
| # ==================== Step 1: 构建并推送 ==================== | |||
| if [ "$NO_BUILD" = false ]; then | |||
| section "Step 1/2: 构建并推送镜像" | |||
| cd "$PROJECT_DIR" | |||
| BRANCH=$(git rev-parse --abbrev-ref HEAD | sed 's/\//-/g') | |||
| TAG="$(date +%Y%m%d%H%M)-${BRANCH}" | |||
| COMMIT_SHORT=$(git rev-parse --short HEAD) | |||
| GIT_DIRTY="" | |||
| git diff --quiet && git diff --cached --quiet 2>/dev/null || GIT_DIRTY="-dirty" | |||
| TAG="$(date +%Y%m%d%H%M)-${BRANCH}-${COMMIT_SHORT}${GIT_DIRTY}" | |||
| IMAGE="${DOCKER_REGISTRY}:${TAG}" | |||
| info "分支: ${BRANCH}" | |||
| @@ -96,7 +94,6 @@ else | |||
| ok "使用镜像: ${IMAGE}" | |||
| fi | |||
| # ==================== Step 2: 部署到测试节点 ==================== | |||
| section "Step 2/2: 部署到测试环境" | |||
| FAILED=() | |||
| @@ -104,6 +101,7 @@ FAILED=() | |||
| deploy_local() { | |||
| local NAME="$1" | |||
| local DIR="$2" | |||
| local NODE_TYPE_VALUE="$3" | |||
| echo "" | |||
| info "部署到 ${NAME}..." | |||
| @@ -120,6 +118,11 @@ deploy_local() { | |||
| echo " [1/3] 更新 image tag..." | |||
| sed -i "s|${DOCKER_REGISTRY}:.*|${DOCKER_REGISTRY}:${TAG}|" docker-compose.yml | |||
| if grep -q "NODE_TYPE=" docker-compose.yml; then | |||
| sed -i "s|NODE_TYPE=.*|NODE_TYPE=${NODE_TYPE_VALUE}|" docker-compose.yml | |||
| else | |||
| sed -i "/SESSION_NAME=/a\\ - NODE_TYPE=${NODE_TYPE_VALUE}" docker-compose.yml | |||
| fi | |||
| echo " [2/3] 拉取新镜像..." | |||
| docker compose pull new-api | |||
| @@ -133,15 +136,18 @@ deploy_local() { | |||
| } | |||
| case "$TARGET" in | |||
| cn) deploy_local "国内测试(CN)" "$CN_DIR" ;; | |||
| ov) deploy_local "国外测试(OV)" "$OV_DIR" ;; | |||
| cn) | |||
| deploy_local "国内测试(CN)" "$CN_DIR" "master" | |||
| ;; | |||
| ov) | |||
| deploy_local "国外测试(OV)" "$OV_DIR" "slave" | |||
| ;; | |||
| all) | |||
| deploy_local "国内测试(CN)" "$CN_DIR" | |||
| deploy_local "国外测试(OV)" "$OV_DIR" | |||
| deploy_local "国内测试(CN)" "$CN_DIR" "master" | |||
| deploy_local "国外测试(OV)" "$OV_DIR" "slave" | |||
| ;; | |||
| esac | |||
| # ==================== 总结 ==================== | |||
| echo "" | |||
| section "部署总结" | |||
| echo -e " 镜像: ${GREEN}${IMAGE}${NC}" | |||
| @@ -72,7 +72,10 @@ if [ "$NO_BUILD" = false ]; then | |||
| cd "$PROJECT_DIR" | |||
| BRANCH=$(git rev-parse --abbrev-ref HEAD | sed 's/\//-/g') | |||
| TAG="$(date +%Y%m%d%H%M)-${BRANCH}" | |||
| COMMIT_SHORT=$(git rev-parse --short HEAD) | |||
| GIT_DIRTY="" | |||
| git diff --quiet && git diff --cached --quiet 2>/dev/null || GIT_DIRTY="-dirty" | |||
| TAG="$(date +%Y%m%d%H%M)-${BRANCH}-${COMMIT_SHORT}${GIT_DIRTY}" | |||
| IMAGE="${DOCKER_REGISTRY}:${TAG}" | |||
| info "分支: ${BRANCH}" | |||
| @@ -127,7 +130,18 @@ echo " [1/4] 更新 image tag..." | |||
| sed -i "s|${REGISTRY}:.*|${REGISTRY}:${TAG}|" docker-compose.yml | |||
| echo " [2/4] 拉取新镜像..." | |||
| docker compose pull new-api | |||
| PULL_TIMEOUT=120 | |||
| for i in 1 2 3; do | |||
| if timeout "${PULL_TIMEOUT}" docker compose pull --quiet new-api; then | |||
| break | |||
| fi | |||
| if [ "$i" -eq 3 ]; then | |||
| echo " [2/4] 拉取失败,已重试 3 次(每次 ${PULL_TIMEOUT}s 超时)" | |||
| exit 1 | |||
| fi | |||
| echo " [2/4] 拉取超时或失败(${PULL_TIMEOUT}s),第 $((i+1)) 次重试..." | |||
| sleep 5 | |||
| done | |||
| echo " [3/4] 重启服务..." | |||
| docker compose up -d new-api | |||