Observability (Backend Standard)

This document defines the minimum observability bar for Flow Create Solutions backend services.

Scope:

• Correlation IDs (X-Request-Id)
• Structured logging (required fields + redaction)
• Metrics (Prometheus-style naming)
• Tracing (OpenTelemetry; optional until wired)

Correlation IDs (required)

Header standard

• Canonical header: X-Request-Id
• Inbound compatibility: accept X-Correlation-Id but map it to X-Request-Id internally.
• Outbound behavior: always emit X-Request-Id on responses.

Generation and propagation rules

• If the inbound request provides X-Request-Id, use it (after basic validation/length bounds).
• Else if the inbound request provides X-Correlation-Id, use it as the request id (same validation/length bounds).
• Else generate a new request id (UUID v4 recommended).
• Store the value for the duration of the request (e.g., request.state.request_id).
• Include it in:
  • logs (every log line for that request)
  • outbound HTTP calls (as X-Request-Id)
  • responses (as X-Request-Id)

Example (FastAPI middleware)

from __future__ import annotations

from uuid import uuid4

from fastapi import FastAPI, Request
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.responses import Response


def _normalize_request_id(value: str | None) -> str | None:
    if not value:
        return None
    value = value.strip()
    # Prevent abuse / huge headers; keep it small and log-friendly.
    if len(value) > 128:
        return None
    return value


class RequestIdMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request: Request, call_next):
        request_id = _normalize_request_id(request.headers.get("X-Request-Id"))
        if not request_id:
            request_id = _normalize_request_id(request.headers.get("X-Correlation-Id"))
        if not request_id:
            request_id = str(uuid4())

        request.state.request_id = request_id

        response: Response = await call_next(request)
        response.headers["X-Request-Id"] = request_id
        return response


def install_observability(app: FastAPI) -> None:
    app.add_middleware(RequestIdMiddleware)

Structured logging (required)

Baseline requirements

• Logs must be structured in production (JSON preferred).
• Every request log and request-scoped log line must include:
  • request_id (from X-Request-Id)
  • route (normalized route template if available, e.g. /api/v1/items/{id})
  • method
  • status_code (for request summary logs)
  • duration_ms (for request summary logs)

Redaction rules (non-negotiable)

• Never log:
  • passwords or secrets
  • full JWTs (log last 4 chars at most)
  • API keys
  • full request bodies unless explicitly reviewed and redacted
• Treat tenant identifiers as operational metadata; do not log sensitive user PII as “labels”.

Logger usage guidance

• Use logger.exception(...) only when you want a traceback.
• Prefer a single request “summary log” per request (method/route/status/duration/request_id).

Metrics (recommended baseline; Prometheus-style)

Naming conventions

• Use lowercase with underscores.
• Prefer suffixes:
  • _total for counters (e.g., http_requests_total)
  • _seconds for durations (e.g., http_request_duration_seconds)

Cardinality rules

• Avoid high-cardinality labels (user ids, raw paths, emails, request ids).
• Acceptable labels typically include:
  • method
  • route (template, not raw path)
  • status_code
  • service / environment (if you emit multi-service metrics)

Minimum recommended HTTP metrics

• http_requests_total{method,route,status_code}
• http_request_duration_seconds{method,route} (histogram)
• http_in_flight_requests{route} (gauge; optional)

Tracing (OpenTelemetry; optional until wired)

Policy

• Standardize on OpenTelemetry for distributed tracing.
• Tracing is optional until the project has it wired (CI/config/collector/exporter).
• Once wired, each request should create a root span that includes request_id and route/method metadata.

Span naming guidance

• HTTP server span name: HTTP <method> <route> (template route)
• External calls: HTTP <method> <host> (avoid embedding full URLs with ids)
• DB spans should include the operation and table/entity (avoid raw query text if it may contain sensitive data)