| @@ -0,0 +1,236 @@ | |||||
| # Log Chat ID And Upstream ID Design | |||||
| ## Summary | |||||
| This design adds two new usage-log fields: | |||||
| - `chat_id`: extracted from the incoming request body | |||||
| - `upstream_id`: extracted from the upstream response headers | |||||
| The existing `logs.request_id` field keeps its current meaning and continues to represent the internal new-api request ID. | |||||
| The extraction logic is centralized and writes normalized values into relay context first, then both consume logs and error logs read from the same context when persisting to `logs`. | |||||
| ## Problem | |||||
| The current system has only one stable request identifier in usage logs: the internal `logs.request_id`. | |||||
| For reconciliation work, that is not enough: | |||||
| - the business request may already carry a `chat_id` | |||||
| - the upstream provider may return its own request ID in response headers | |||||
| Today these values are not recorded consistently in consume logs. Error logs have partial upstream request-id support, but consume logs do not have a unified path. | |||||
| ## Goals | |||||
| - Persist request-body `chat_id` into usage logs | |||||
| - Persist upstream response request ID into usage logs | |||||
| - Keep success and error logs consistent | |||||
| - Keep the extraction logic generic enough to support future upstream header changes | |||||
| - Keep streaming support without buffering the full stream body | |||||
| - Keep performance impact negligible | |||||
| ## Non-Goals | |||||
| - No change to the meaning of existing `logs.request_id` | |||||
| - No backfill of existing historical logs | |||||
| - No first-version parsing of response body IDs | |||||
| - No provider-specific per-channel logging branches unless the generic extractor cannot cover them | |||||
| ## Canonical Field Semantics | |||||
| The `logs` table will have three distinct request identifiers: | |||||
| - `request_id`: internal new-api request ID, already existing | |||||
| - `chat_id`: business ID extracted from the incoming request body | |||||
| - `upstream_id`: upstream request ID extracted from response headers | |||||
| Field semantics must not overlap. | |||||
| ## Data Model | |||||
| Add two nullable string columns to `logs`: | |||||
| - `chat_id` | |||||
| - `upstream_id` | |||||
| Constraints: | |||||
| - both fields default to empty string | |||||
| - both fields use ordinary indexes | |||||
| - neither field is unique | |||||
| Reasoning: | |||||
| - reconciliation queries need direct filtering and export | |||||
| - uniqueness cannot be guaranteed across providers or retries | |||||
| ## Extraction Architecture | |||||
| ### 1. Request-side extraction | |||||
| When the request body is parsed into the relay request object, the system performs a best-effort extraction of a top-level `chat_id`. | |||||
| Behavior: | |||||
| - extract only the business-level request `chat_id` | |||||
| - do not deeply traverse nested objects | |||||
| - do not fail the request if `chat_id` is missing | |||||
| - store the result in relay context as `RelayInfo.ChatID` | |||||
| ### 2. Response-side extraction | |||||
| When an upstream `http.Response` is received, before body consumption begins, the system performs a best-effort extraction of the upstream request ID and stores it in `RelayInfo.UpstreamID`. | |||||
| First-version default rule chain: | |||||
| 1. `x-request-id` | |||||
| 2. `request-id` | |||||
| This rule chain is intentionally generic. The extractor should be implemented as a reusable module with: | |||||
| - a default candidate-header chain | |||||
| - a future extension point for provider or channel-specific overrides | |||||
| ### 3. Persistence | |||||
| Consume logs and error logs do not parse request bodies or response headers directly. | |||||
| They only read: | |||||
| - internal `request_id` | |||||
| - `RelayInfo.ChatID` | |||||
| - `RelayInfo.UpstreamID` | |||||
| and persist them into `logs`. | |||||
| This keeps extraction and logging decoupled. | |||||
| ## Data Flow | |||||
| ### Non-stream requests | |||||
| 1. request enters gateway | |||||
| 2. internal request ID is created as today | |||||
| 3. request parser extracts `chat_id` | |||||
| 4. relay runs normally | |||||
| 5. upstream response is received | |||||
| 6. response extractor reads `x-request-id` / `request-id` | |||||
| 7. final consume log or error log persists `request_id`, `chat_id`, `upstream_id` | |||||
| ### Stream requests | |||||
| 1. request parser extracts `chat_id` before relay starts | |||||
| 2. upstream response headers are received before stream body forwarding | |||||
| 3. response extractor reads `upstream_id` from headers | |||||
| 4. stream body is forwarded as usual | |||||
| 5. final consume log or error log persists `request_id`, `chat_id`, `upstream_id` | |||||
| No per-chunk ID parsing is required in the first version. | |||||
| ## Retry Semantics | |||||
| Retries must not collapse multiple upstream attempts into one combined ID set. | |||||
| Rules: | |||||
| - consume log records only the final successful attempt's `upstream_id` | |||||
| - each failed attempt may still produce its own error log with its own `upstream_id` | |||||
| - the final consume log should not store a list of retry upstream IDs | |||||
| This preserves clean reconciliation semantics for billed requests. | |||||
| ## Performance Requirements | |||||
| The design must not materially affect relay throughput or stream latency. | |||||
| Allowed work: | |||||
| - read response headers once | |||||
| - best-effort request-side `chat_id` extraction while request parsing already happens | |||||
| Forbidden first-version approaches: | |||||
| - buffering the full response body only to find IDs | |||||
| - parsing every SSE chunk to search for IDs | |||||
| - adding extra database writes per request | |||||
| Expected impact: | |||||
| - request-side `chat_id` extraction: negligible, because request parsing already occurs | |||||
| - response-side `upstream_id` extraction: negligible, because headers are already available | |||||
| - database overhead: one existing log insert with two extra fields | |||||
| ## Error Handling | |||||
| The feature is best-effort only. | |||||
| Rules: | |||||
| - missing `chat_id` does not fail the request | |||||
| - missing upstream header does not fail the request | |||||
| - malformed request payload does not add extra parsing failure beyond current validation behavior | |||||
| - logs may store empty values for either field | |||||
| ## Query And UI Scope | |||||
| First version should support: | |||||
| - backend filtering by `chat_id` | |||||
| - backend filtering by `upstream_id` | |||||
| - frontend display in log detail or an equivalent low-risk display path | |||||
| The first version does not need both fields as default main table columns. | |||||
| ## Testing Scope | |||||
| ### Backend tests | |||||
| - non-stream success: request body includes `chat_id`, upstream returns `x-request-id`, consume log stores both | |||||
| - stream success: request body includes `chat_id`, upstream stream response includes `x-request-id`, consume log stores both | |||||
| - error response: error log stores both when available | |||||
| - retry then success: consume log stores only the final successful `upstream_id` | |||||
| - missing fields: request still succeeds or fails normally, log fields remain empty | |||||
| - header fallback: if `x-request-id` is absent and `request-id` exists, `upstream_id` is still stored | |||||
| ### Regression focus | |||||
| - no change to existing `logs.request_id` behavior | |||||
| - no extra stream buffering | |||||
| - no change to billing semantics | |||||
| ## Implementation Notes | |||||
| Recommended implementation shape: | |||||
| - extend `relay/common.RelayInfo` with `ChatID` and `UpstreamID` | |||||
| - add a small extraction helper for request-side `chat_id` | |||||
| - add a reusable response-header extractor for upstream IDs | |||||
| - extend `model.Log` and log persistence functions to carry `chat_id` and `upstream_id` | |||||
| - update log query/filter surfaces to support the two new fields | |||||
| ## Risks | |||||
| - some providers may later rename or stop returning `x-request-id` | |||||
| - some request formats may not include top-level `chat_id` | |||||
| - stream and retry paths can drift if the extractor is not wired at a shared layer | |||||
| Mitigation: | |||||
| - use one shared response extractor with a candidate chain | |||||
| - use one shared relay context as the single source of truth | |||||
| - test both normal and retry flows explicitly | |||||
| ## Decision Summary | |||||
| Approved design decisions: | |||||
| - keep `logs.request_id` unchanged as the internal request ID | |||||
| - add `logs.chat_id` | |||||
| - add `logs.upstream_id` | |||||
| - extract `chat_id` from the request body | |||||
| - extract `upstream_id` from response headers, defaulting to `x-request-id` and falling back to `request-id` | |||||
| - centralize extraction into shared relay context | |||||
| - support both stream and non-stream requests without full-body buffering | |||||
| - record only the final successful `upstream_id` in consume logs during retry scenarios | |||||