World map

 

BM

World O4: The Diary & One Order

Exemplars and correlation

You will learn: to jump from a spike on a graph to the very order that caused it.

Checked with Prometheus 3 on 1 Nov 2026

The problem

Press Run. Which service got slow, and when?

Metrics in this data (tap to add):

Time 0 is when the data starts. The query looks back from the eval time.

The graph shows kedai-pay getting slow from 10m. But which order was slow? A graph can’t name one.

Watch

What the pay service exposes, in OpenMetrics text. Tap the part that names one real order.

0 of 1 found

An exemplar: A sample trace ID pinned to a metric point. Click the dot on the graph, open that very order's trace. pins one sample order to a metric. Grafana draws it as a dot you can click.

Try it

Where you start at the top. Put these in order. Use the arrows, or drag.

  • Exemplar dot: trace 3c9a51e0…
  • Trace: SELECT payments took 2.8 s
  • Graph: p95 of kedai-pay jumps at 10m
  • Logs with that trace_id: slow query on payments

Put the jumps in order, from the first clue to the reason.

Predict

Prometheus does not keep exemplars by default. Which flag turns them on?

Check

Drag a card into a box. Or tap the card, then tap the box.

        Which signal answers each question best?

        You did it!

        You can now jump from a graph to one order’s trace and logs.

        Go deeper (optional)

        In Grafana, the Prometheus data source can link an exemplar’s trace_id to your trace store. A Loki data source can turn trace_id in a log line into a link too. Then the jump is one click, both ways.