Distributed tracing: definition
Distributed tracing records the path of a request or transaction as it moves through services and operations. A trace is composed of spans, where each span represents a timed unit of work with attributes, status and relationships to other spans.
Why tracing is useful
In a distributed application, end-to-end latency may be the sum of many service calls. A trace can reveal which span consumed time, where an error appeared and which downstream dependency was involved. This makes tracing especially useful when a metric identifies a symptom but not the request path behind it.
Context propagation
Services need to propagate trace context so spans from different processes can be assembled into one trace. Standards such as W3C Trace Context help different libraries and platforms carry a common trace identity across boundaries.
Span attributes and events
Useful spans include stable attributes such as service, operation, route and dependency information. Avoid attaching unbounded or sensitive values indiscriminately. Trace data can be powerful precisely because it is detailed, so privacy and cost controls matter.
Sampling
High-volume services often sample traces. Head sampling decides early, while tail-based approaches can consider information observed later in the trace. The sampling strategy should preserve enough rare errors and slow requests to support the investigations you care about.
Common tracing failures
- Broken context propagation creates disconnected traces.
- Inconsistent service names make topology hard to read.
- Missing database or external-call spans hide important dependencies.
- Aggressive sampling drops the rare incidents responders need most.
- Sensitive attributes are captured without a retention or access policy.
Related concepts
Short answer
Distributed tracing follows a request across services using related spans. It is most useful when context propagates reliably, spans contain meaningful attributes and sampling preserves the failures and latency patterns you need to investigate.
Practical checklist
- Verify context propagation across every important boundary.
- Keep span names and service names stable and interpretable.
- Sample in a way that preserves rare errors and slow requests.
- Avoid sensitive or unbounded attributes in spans.