keythread

Deterministic cross-tool trace correlation for DORA metrics.

Webhooks carry no trace context. When an issue tracker, a git host, and a CI/CD system each fire independent webhooks for the same piece of work, there's normally no way to link them into one trace — so metrics like lead time for changes can't be computed automatically across tool boundaries.

keythread closes that gap with one technique: hash a business/correlation key (an issue key, a ticket ID — whatever your tracking system uses) into a deterministic OpenTelemetry trace ID with HMAC-SHA256. Every webhook that carries the same key lands in the same trace, regardless of which tool sent it or when.

issue key "PROJ-123"
        │
        ▼
HMAC-SHA256(secret, "proj-123")  ──►  same 128-bit trace ID every time
        │
        ├── Jira: issue created         ─┐
        ├── GitLab: commit pushed        ├─► all land in one OTel trace
        └── GitLab: deployment completed ─┘

Install

Not yet published to PyPI — install from source for now:

# library only
pip install "keythread @ git+https://github.com/NewerKey/keythread"

# + the webhook server (FastAPI/uvicorn/typer)
pip install "keythread[server] @ git+https://github.com/NewerKey/keythread"

# + OTLP/HTTP export instead of the default gRPC exporter
pip install "keythread[server,otlp-http] @ git+https://github.com/NewerKey/keythread"

Once published, these will become pip install keythread, pip install "keythread[server]", etc.

Quickstart (server)

cp .env.example .env
# fill in TRACE_ID_HMAC_SECRET / GITLAB_WEBHOOK_SECRET / JIRA_WEBHOOK_SECRET
# (there are no insecure defaults — the server refuses to start without them)

keythread serve                          # binds 0.0.0.0:8000 by default
keythread serve --host 127.0.0.1 --port 9000

keythread version                        # print the installed version
keythread --help / keythread serve --help

(From a clone rather than a pip install, prefix each command with uv run.)

curl localhost:8000/health/live
curl localhost:8000/health/ready
curl localhost:8000/metrics   # includes dora_lead_time_for_change_seconds

Point your Jira Automation rule / GitLab webhook config at POST /webhooks/jira and POST /webhooks/gitlab respectively, with the matching secret header (X-Webhook-Secret / X-Gitlab-Token). For any other tool (GitHub, Linear, ...), see Writing a new adapter below.

Quickstart (library)

from keythread import (
    JiraAdapter, GitLabAdapter, register_adapter,
    SpanFactory, WorkflowStateManager, Settings, setup_telemetry,
    process_webhook_event,
)

settings = Settings()  # reads TRACE_ID_HMAC_SECRET etc. from the environment

register_adapter("jira", JiraAdapter(settings.jira_webhook_secret, settings.trace_id_hmac_secret))
register_adapter("gitlab", GitLabAdapter(settings.gitlab_webhook_secret, settings.trace_id_hmac_secret))

tracer = setup_telemetry(settings)
span_factory = SpanFactory(tracer)
state_manager = WorkflowStateManager()

process_webhook_event("jira", jira_payload, span_factory, state_manager)
process_webhook_event("gitlab", gitlab_push_payload, span_factory, state_manager)
process_webhook_event("gitlab", gitlab_deploy_payload, span_factory, state_manager)

Writing a new adapter

SourceAdapter is a typing.Protocol — implement three methods, no subclassing required:

from keythread import SourceAdapter, UnifiedEvent, register_adapter

class LinearAdapter:
    def verify_request(self, headers, body) -> bool: ...
    def extract_correlation_key(self, payload: dict) -> str | None: ...
    def map_event(self, payload: dict) -> UnifiedEvent | None: ...

register_adapter("linear", LinearAdapter())

That registers it explicitly, for your own app. To distribute an adapter as an installable package instead — so any team can add support for your tool with pip install, not a PR into this repo — publish it as a keythread_adapters entry point (mirroring OpenTelemetry's own opentelemetry_instrumentor entry-point group, which is how opentelemetry-bootstrap auto-discovers instrumentation libraries):

# your package's pyproject.toml
[project.entry-points.keythread_adapters]
linear = "keythread_linear:build_adapter"

# keythread_linear/__init__.py
import os
from keythread import SourceAdapter

def build_adapter() -> SourceAdapter:
    """Called with no arguments — read whatever this adapter needs
    (its own webhook secret, the shared TRACE_ID_HMAC_SECRET, ...)
    from the environment itself."""
    return LinearAdapter(
        webhook_secret=os.environ["LINEAR_WEBHOOK_SECRET"],
        trace_id_hmac_secret=os.environ["TRACE_ID_HMAC_SECRET"],
    )

create_app (or a direct call to keythread.discover_adapters()) picks this up automatically at startup — see docs/otel-extensibility.md for the full comparison against OpenTelemetry's own extensibility model. A plugin that fails to import or construct (e.g. its secret isn't set because this deployment doesn't use that tool) is logged and skipped, not fatal.

Any object satisfying those three methods works — SourceAdapter is structural, not nominal. adapters/github.py is a real (not sketched) second built-in adapter and the reference to copy from — it deliberately differs from Jira/GitLab in ways worth studying before writing your own:

• Auth scheme: GitHub signs the raw request body with HMAC-SHA256 (X-Hub-Signature-256), unlike Jira/GitLab's plain shared-secret header — verify_request receives body: bytes for exactly this reason.
• No native issue-key format: GitHub has no Jira-style PROJ-123; correlation keys are derived from issue numbers (#42, branch names like 42-fix-thing), normalized to GH-42.
• Event kind isn't in the payload: GitHub sends it via the X-GitHub-Event header, but map_event only receives the body — the adapter infers kind from payload shape instead (see _infer_event_kind), a real limitation worth knowing about if your tool's payloads are ambiguous without their dispatch header.

To run the FastAPI server with a custom adapter instead of (or alongside) the Jira/GitLab built-ins, pass it to create_app:

import uvicorn
from keythread.adapters.github import GitHubAdapter
from keythread.server.app import create_app
from keythread.settings import Settings

settings = Settings()
app = create_app(
    settings,
    adapters={
        "github": GitHubAdapter(
            webhook_secret="...",  # GitHub webhook's configured secret
            trace_id_hmac_secret=settings.trace_id_hmac_secret,
        )
    },
)
uvicorn.run(app)

A key that collides with a built-in ("jira" or "gitlab") replaces it — useful for swapping in a customized adapter without forking keythread. The keythread serve CLI command itself only wires up the Jira/GitLab built-ins; teams on other tools use create_app directly as shown above (a few lines, not a fork) instead of the CLI entry point.

Architecture

Module	Responsibility
correlation.py	HMAC-SHA256 trace/span ID generation
schema.py	UnifiedEvent, EventType (closed enum of DORA lifecycle stages)
id_generator.py	DeterministicIdGenerator — injects trace/span IDs into OTel via a ContextVar. A custom OTel IdGenerator, not a Propagator — see the module docstring
telemetry.py	setup_telemetry — accepts an injected TracerProvider for testing
span_factory.py	Maps a UnifiedEvent to an OTel span with semantic-convention attributes
state.py	WorkflowStateManager — trace-aware commit/deploy correlation state, TTL + lazy eviction
metrics.py	Prometheus DORA metrics (deployment frequency, lead time)
payload_sink.py	Opt-in raw-payload persistence (no-op by default)
adapters/	SourceAdapter protocol, registry (+ keythread_adapters entry-point plugin discovery), built-in Jira/GitLab/GitHub adapters
webhooks/	Request verification + processing orchestration
server/	Optional FastAPI app (server extra)

telemetry.py::build_tracer_provider() configures a TracerProvider + BatchSpanProcessor + OTLPSpanExporter + a custom Resource and IdGenerator — the same shape an OTel "distro" package configures. It isn't registered under the opentelemetry_distro/opentelemetry_configurator entry points, though, so it can't be picked up by the zero-code opentelemetry-instrument CLI the way opentelemetry-distro can. Deliberate, not missing: keythread's primary surface is its own webhook server, not an arbitrary third-party app being auto-instrumented, so zero-code activation isn't a use case that applies here. See docs/otel-extensibility.md for the full comparison against OpenTelemetry's documented extensibility model.

Security

• Fail closed: Settings requires TRACE_ID_HMAC_SECRET, GITLAB_WEBHOOK_SECRET, and JIRA_WEBHOOK_SECRET — the process refuses to start if any are unset. There are no placeholder fallbacks.
• Constant-time comparisons: webhook credential checks use hmac.compare_digest.
• Opt-in disk writes: payloads are never written to disk unless you explicitly configure a FilePayloadSink.

Development

uv sync --all-extras --dev
uv run ruff check .
uv run ruff format --check .
uv run pytest --cov=src --cov-report=term-missing

Status & contributing

This is active, ongoing work, not a finished v1 — Jira/GitLab/GitHub are the only built-in adapters so far, and design decisions (the correlation key concept, the TTL-based state manager, what belongs in the core vs. an adapter) are still being stress-tested against real tools. Feedback on any of that is genuinely wanted, not a formality:

• Found a bug, or a design decision that doesn't hold up for your tool? Open an issue.
• Wrote an adapter for a tool not listed here (GitHub Issues + Actions, Linear, Azure DevOps, Jenkins, CircleCI, Bitbucket, ...)? A PR into this repo is welcome — adapters/github.py is the reference to copy from, and the existing adapter test suites (tests/adapters/) show the coverage a new one is expected to have. You don't have to go through core, though: publish it as a keythread_adapters entry point (see Writing a new adapter) and it's picked up automatically, no PR required.
• Disagree with a tradeoff (the dropped CORRELATION_STRATEGY toggle, the TTL default, the closed EventType enum, anything in Security)? Say so in an issue — these were judgment calls made with the information available at the time, not settled conclusions.

License

MIT — see LICENSE.