World map

 

BM

World O4: The Diary & One Order

Loki and LogQL basics

You will learn: to pick a diary by its labels and filter its lines with LogQL.

Checked with Prometheus 3 on 1 Nov 2026

The problem

Five services keep five diaries, on five machines. A payment timed out somewhere. Where do you start?

Watch

bash
logcli labels
app
service_name
logcli labels app
kedai-api
kedai-pay

Loki: Grafana's log store. It indexes only the labels of each diary, not every word, so it stays small and cheap. keeps every diary on one shelf. It indexes only the labels, like app. Not every word.

Try it

Ask for kedai-api’s lines that contain “timeout”. Tap the pieces in order.

Tap the pieces in the right order.

…

Predict

Which lines does {app="kedai-api"} | json | level="error" return?

0 of 1 found

LogQL: Loki's query language. Pick diaries by label in braces, then filter lines: {app="kedai-api"} |= "timeout". reads like a pipe. | json turns fields into labels. Then you can filter on them.

Check

bash
logcli query --quiet '{app="kedai-api"} |= "timeout"'
2026-11-01T17:45:22Z {} {"level":"error","msg":"payment timeout","order_id":"A1041","duration_ms":3000,"trace_id":"3c9a51e0b7d24f6e9a18c2d4e6f80b15"}
logcli query --quiet '{app="kedai-api"} | json | level="error"'
2026-11-01T17:45:22Z {} {"level":"error","msg":"payment timeout","order_id":"A1041","duration_ms":3000,"trace_id":"3c9a51e0b7d24f6e9a18c2d4e6f80b15"}

Real Loki 3.7 agrees: one line, two ways to ask. Notice its trace_id. You will need it soon.

Check

parse error : queries require at least one regexp or equality matcher that does not have an empty-compatible value. For instance, app=~".*" does not meet this requirement, but app=~".+" will

You asked for {app=~".*"}: “every diary”. Loki refused. Which fix works?

You did it!

You can now pick diaries by label and filter their lines with LogQL.

Go deeper (optional)

LogQL can count too: sum by (app) (count_over_time({app=~"kedai-.*"} | json | level=~"error|warn" [10m])) counts problem lines per app. Keep labels few and fixed, like app. Never put order_id in a label: each value makes a new log stream: One diary in Loki: all lines with the same set of labels. Too many label values make too many diaries..

Real mission

  1. docker network create logs-try
  2. docker run -d --name loki-try --network logs-try -p 3100:3100 grafana/loki:3.7.9
  3. curl -s -H "Content-Type: application/json" -X POST http://localhost:3100/loki/api/v1/push --data-raw "{\"streams\":[{\"stream\":{\"app\":\"try\"},\"values\":[[\"$(date +%s)000000000\",\"hello loki\"]]}]}"
  4. docker run --rm --network logs-try grafana/logcli:3.7.9 --addr=http://loki-try:3100 query --quiet '{app="try"}'
  5. docker rm -f loki-try && docker network rm logs-try

0 of 5 steps done · Your ticks are saved.