Logs, metrics and traces explained

When a website slows down or a service fails, engineers need to know what changed and where. Logs, metrics, and traces are telemetry: data that software and infrastructure generate as they run. Metrics show patterns, logs record events, and traces follow requests across connected services.

Together, these signals help operators spot problems, locate the affected part of a request, and inspect related events.

What each signal shows

The three signals are often called the “three pillars” of observability, the practice of using a service’s output to understand its behavior. Observability also includes other kinds of telemetry, such as profiles that show where code uses resources. The three signals remain a useful starting point because they describe patterns, events, and request paths.

Metrics reveal patterns

A metric is a measurement recorded over time, such as the number of requests, the rate of failed requests, or CPU use. A graph of request latency can show when a service began slowing down.

Metrics come in different forms. A counter adds up events, such as completed requests. A gauge reports a value at a point in time, such as current memory use. A histogram shows how measurements are spread across values, such as request durations. A histogram can show how a range of requests performed, rather than reducing every request to a single average.

Because metrics summarize many observations, they work well for dashboards and alerts. An alert can notify an operator when an error rate crosses a chosen threshold. A metric points to a change in behavior, but it rarely explains what happened in one particular request.

Logs record events

A log is a record of something that happened, such as a service starting, a payment failing, or a request receiving an error. Logs often include a timestamp and a message. They may also include structured fields, such as severity, service name, or an identifier for the related request.

Structured logs are easier to search and compare than messages written only as free text. For example, an operator can filter for error records from a particular service. Logs provide detail about an event, but a log by itself may not show how a request moved between services.

Because logs can contain sensitive information, operators should decide what to record and where to store it. Collection tools can transform or remove fields, but the right safeguards depend on the data and the applicable privacy requirements.

Traces follow requests

A trace follows one request or operation through the parts of a service that handled it. A trace is made up of spans. Each span describes an operation, how long it took, and how it relates to other operations.

Imagine a page request that passes through an API gateway, an application service, and a database. A trace can show the duration of each step, making it easier to see whether the delay came from the database call or another part of the path. Trace details depend on which operations engineers configure the software to record.

Some systems sample traces to limit how much data they collect and process. With head sampling, the system decides early whether to keep a trace. Tail sampling makes that decision after the trace has finished, which allows rules to consider details such as whether the trace was slow or contained an error. Sampling means that a stored trace set may not include every request.

Use logs, metrics, and traces together

Suppose a service’s latency metric rises. The metric shows an operator that response times have increased. A trace can then show which service or operation contributed to the delay. Logs linked to that trace can provide details about errors, retries, or other events during the request.

That link depends on shared context. A log record with the trace’s identifier lets an operator find the related trace. Without consistent identifiers and compatible instrumentation, logs, metrics, and traces remain separate clues. Some events, such as infrastructure messages or older logs, may not include request context.

Choose what to collect

Start with the questions operators need to answer. A service that handles customer requests might track request volume, error rate, and latency as metrics. It might record detailed error events in logs and create traces that cover the important steps in a request. The specific measurements should match the service’s behavior and the failures that matter.

Use labels and attributes with care. When a metric uses many distinct label values, it creates more time series, or separate sequences of measurements over time. That can make charts slower and data harder to manage. For example, adding a distinct user identifier to a metric can produce a separate series for many users. Keep high-detail event data in an appropriate log or trace field instead of turning every unique value into a metric label.

Choose retention and sampling settings based on how much detail operators need and how much telemetry they can store and process. There is no universal sampling percentage or cost ranking for the three signals. Those choices depend on data volume, retention, query needs, and the tools in use.