| @@ -0,0 +1,236 @@ | |||
| # Log Chat ID And Upstream ID Design | |||
| ## Summary | |||
| This design adds two new usage-log fields: | |||
| - `chat_id`: extracted from the incoming request body | |||
| - `upstream_id`: extracted from the upstream response headers | |||
| The existing `logs.request_id` field keeps its current meaning and continues to represent the internal new-api request ID. | |||
| The extraction logic is centralized and writes normalized values into relay context first, then both consume logs and error logs read from the same context when persisting to `logs`. | |||
| ## Problem | |||
| The current system has only one stable request identifier in usage logs: the internal `logs.request_id`. | |||
| For reconciliation work, that is not enough: | |||
| - the business request may already carry a `chat_id` | |||
| - the upstream provider may return its own request ID in response headers | |||
| Today these values are not recorded consistently in consume logs. Error logs have partial upstream request-id support, but consume logs do not have a unified path. | |||
| ## Goals | |||
| - Persist request-body `chat_id` into usage logs | |||
| - Persist upstream response request ID into usage logs | |||
| - Keep success and error logs consistent | |||
| - Keep the extraction logic generic enough to support future upstream header changes | |||
| - Keep streaming support without buffering the full stream body | |||
| - Keep performance impact negligible | |||
| ## Non-Goals | |||
| - No change to the meaning of existing `logs.request_id` | |||
| - No backfill of existing historical logs | |||
| - No first-version parsing of response body IDs | |||
| - No provider-specific per-channel logging branches unless the generic extractor cannot cover them | |||
| ## Canonical Field Semantics | |||
| The `logs` table will have three distinct request identifiers: | |||
| - `request_id`: internal new-api request ID, already existing | |||
| - `chat_id`: business ID extracted from the incoming request body | |||
| - `upstream_id`: upstream request ID extracted from response headers | |||
| Field semantics must not overlap. | |||
| ## Data Model | |||
| Add two nullable string columns to `logs`: | |||
| - `chat_id` | |||
| - `upstream_id` | |||
| Constraints: | |||
| - both fields default to empty string | |||
| - both fields use ordinary indexes | |||
| - neither field is unique | |||
| Reasoning: | |||
| - reconciliation queries need direct filtering and export | |||
| - uniqueness cannot be guaranteed across providers or retries | |||
| ## Extraction Architecture | |||
| ### 1. Request-side extraction | |||
| When the request body is parsed into the relay request object, the system performs a best-effort extraction of a top-level `chat_id`. | |||
| Behavior: | |||
| - extract only the business-level request `chat_id` | |||
| - do not deeply traverse nested objects | |||
| - do not fail the request if `chat_id` is missing | |||
| - store the result in relay context as `RelayInfo.ChatID` | |||
| ### 2. Response-side extraction | |||
| When an upstream `http.Response` is received, before body consumption begins, the system performs a best-effort extraction of the upstream request ID and stores it in `RelayInfo.UpstreamID`. | |||
| First-version default rule chain: | |||
| 1. `x-request-id` | |||
| 2. `request-id` | |||
| This rule chain is intentionally generic. The extractor should be implemented as a reusable module with: | |||
| - a default candidate-header chain | |||
| - a future extension point for provider or channel-specific overrides | |||
| ### 3. Persistence | |||
| Consume logs and error logs do not parse request bodies or response headers directly. | |||
| They only read: | |||
| - internal `request_id` | |||
| - `RelayInfo.ChatID` | |||
| - `RelayInfo.UpstreamID` | |||
| and persist them into `logs`. | |||
| This keeps extraction and logging decoupled. | |||
| ## Data Flow | |||
| ### Non-stream requests | |||
| 1. request enters gateway | |||
| 2. internal request ID is created as today | |||
| 3. request parser extracts `chat_id` | |||
| 4. relay runs normally | |||
| 5. upstream response is received | |||
| 6. response extractor reads `x-request-id` / `request-id` | |||
| 7. final consume log or error log persists `request_id`, `chat_id`, `upstream_id` | |||
| ### Stream requests | |||
| 1. request parser extracts `chat_id` before relay starts | |||
| 2. upstream response headers are received before stream body forwarding | |||
| 3. response extractor reads `upstream_id` from headers | |||
| 4. stream body is forwarded as usual | |||
| 5. final consume log or error log persists `request_id`, `chat_id`, `upstream_id` | |||
| No per-chunk ID parsing is required in the first version. | |||
| ## Retry Semantics | |||
| Retries must not collapse multiple upstream attempts into one combined ID set. | |||
| Rules: | |||
| - consume log records only the final successful attempt's `upstream_id` | |||
| - each failed attempt may still produce its own error log with its own `upstream_id` | |||
| - the final consume log should not store a list of retry upstream IDs | |||
| This preserves clean reconciliation semantics for billed requests. | |||
| ## Performance Requirements | |||
| The design must not materially affect relay throughput or stream latency. | |||
| Allowed work: | |||
| - read response headers once | |||
| - best-effort request-side `chat_id` extraction while request parsing already happens | |||
| Forbidden first-version approaches: | |||
| - buffering the full response body only to find IDs | |||
| - parsing every SSE chunk to search for IDs | |||
| - adding extra database writes per request | |||
| Expected impact: | |||
| - request-side `chat_id` extraction: negligible, because request parsing already occurs | |||
| - response-side `upstream_id` extraction: negligible, because headers are already available | |||
| - database overhead: one existing log insert with two extra fields | |||
| ## Error Handling | |||
| The feature is best-effort only. | |||
| Rules: | |||
| - missing `chat_id` does not fail the request | |||
| - missing upstream header does not fail the request | |||
| - malformed request payload does not add extra parsing failure beyond current validation behavior | |||
| - logs may store empty values for either field | |||
| ## Query And UI Scope | |||
| First version should support: | |||
| - backend filtering by `chat_id` | |||
| - backend filtering by `upstream_id` | |||
| - frontend display in log detail or an equivalent low-risk display path | |||
| The first version does not need both fields as default main table columns. | |||
| ## Testing Scope | |||
| ### Backend tests | |||
| - non-stream success: request body includes `chat_id`, upstream returns `x-request-id`, consume log stores both | |||
| - stream success: request body includes `chat_id`, upstream stream response includes `x-request-id`, consume log stores both | |||
| - error response: error log stores both when available | |||
| - retry then success: consume log stores only the final successful `upstream_id` | |||
| - missing fields: request still succeeds or fails normally, log fields remain empty | |||
| - header fallback: if `x-request-id` is absent and `request-id` exists, `upstream_id` is still stored | |||
| ### Regression focus | |||
| - no change to existing `logs.request_id` behavior | |||
| - no extra stream buffering | |||
| - no change to billing semantics | |||
| ## Implementation Notes | |||
| Recommended implementation shape: | |||
| - extend `relay/common.RelayInfo` with `ChatID` and `UpstreamID` | |||
| - add a small extraction helper for request-side `chat_id` | |||
| - add a reusable response-header extractor for upstream IDs | |||
| - extend `model.Log` and log persistence functions to carry `chat_id` and `upstream_id` | |||
| - update log query/filter surfaces to support the two new fields | |||
| ## Risks | |||
| - some providers may later rename or stop returning `x-request-id` | |||
| - some request formats may not include top-level `chat_id` | |||
| - stream and retry paths can drift if the extractor is not wired at a shared layer | |||
| Mitigation: | |||
| - use one shared response extractor with a candidate chain | |||
| - use one shared relay context as the single source of truth | |||
| - test both normal and retry flows explicitly | |||
| ## Decision Summary | |||
| Approved design decisions: | |||
| - keep `logs.request_id` unchanged as the internal request ID | |||
| - add `logs.chat_id` | |||
| - add `logs.upstream_id` | |||
| - extract `chat_id` from the request body | |||
| - extract `upstream_id` from response headers, defaulting to `x-request-id` and falling back to `request-id` | |||
| - centralize extraction into shared relay context | |||
| - support both stream and non-stream requests without full-body buffering | |||
| - record only the final successful `upstream_id` in consume logs during retry scenarios | |||