Observability and Provenance

Observability and provenance describe the evidence by which observers can inspect, explain, correlate, and diagnose system behavior across boundaries and time.

Observability records evidence about executions. It is distinct from the categorical trace and feedback principle, which models outputs becoming future inputs. Provenance records where a value, decision, route, effect, or observation came from and which definitions, inputs, policies, authorities, and occurrences contributed to it.

Observation Scopes

• An operation trace follows one bounded invocation, activation, attempt, or local commit.
• A message history records emission, routing, transformation, delivery, acknowledgment, retry, and disposition evidence.
• A causal chain links occurrences through explicit causation rather than correlation alone.
• A process history explains durable progress across operations, messages, waits, timers, compensations, and human work.
• A system health view aggregates rates, latency, backlog, saturation, errors, expiry, quarantine, and recovery state.

A wire tap or diagnostic subscriber creates another observation path. It must not silently change delivery cardinality, ordering, backpressure, privacy, or failure behavior. A message store used for diagnosis is not automatically authoritative domain history and requires its own retention, access, redaction, and integrity rules.

Service-level evidence uses declared observables and observations to evaluate consumer-visible outcomes over a defined population and window. A health metric is not automatically a service-level indicator: the definition must say which service boundary, operation, consumer scope, eligibility rule, outcome, unit, and aggregation it represents. Instrumentation supplies evidence; it does not choose the objective or create an agreement between provider and consumer.

Observation Contracts and Assurance

An observation used as architectural evidence needs an observation contract. The contract identifies the semantic actions, states, effects, or outcomes to be observed; the subject and boundary; correlation and causation identities; required ordering or timing information; collection coverage; sampling and loss behavior; retention; provenance; and the consequence of missing evidence.

Observability supplies evidence but does not decide what that evidence establishes. Assurance and evidence interprets observations under an explicit policy and may conclude that a claim is established, conditional, refuted, or unknown. Missing or stale telemetry must not silently become success or failure.

Evidence strength is asymmetric. A valid observed counterexample can refute a safety property within its scope, while a finite violation-free history generally cannot prove that the property holds for every possible execution. Liveness additionally depends on declared progress, failure, timing, and fairness assumptions.

Test messages and synthetic transactions should be identifiable, authorized, and scoped. Their effects must be isolated, reversible, or intentionally real; a synthetic marker alone does not prevent a production consumer from performing an irreversible action.

Useful provenance may include definition and semantic revision, node and branch identity, message and request identity, subject and process identity, correlation and causation, route and transformation revisions, source position, handler version, attempt, acknowledgment, commit boundary, authority, and terminal disposition.

Formal relations

• qualifies: Assurance and Evidence — Supplies attributable observations and provenance whose coverage, freshness, and loss modes constrain the assurance conclusions they can support.
• qualifies: Implementation Conformance — Supplies trace and deployment evidence used to compare observed implementation behavior with canonical semantic actions and accepted realization plans.

External References

• Gregor Hohpe and Bobby Woolf, Wire Tap, Message History, and Message Store, Enterprise Integration Patterns, 2003.