World map

 

BM

World O2: Counters

Recording rules

You will learn: how to save a busy query as a new series, name it well and check the rule file.

Checked with Prometheus 3 on 1 Nov 2026

The problem

50 people open the same dashboard. Every 15 s, each one runs the same heavy query. How do you help Prometheus?

Try it

A recording rule: A saved question: Prometheus runs it on a timer and keeps the answer as a new series. saves a query’s answer as a new series. Give this rule its query.

Add an expr line under record, with the per-stall orders query.

rules.yml5 lines

How a computer reads it

  • groupslist · 1 items
    • item 1map · 2 keys
      • namekedaitext
      • ruleslist · 1 items
        • item 1map · 1 keys
          • recordstall:kedai_orders:rate5mtext

Try it

The rule has run. Ask for its saved series, like any other metric.

Show the saved rate for cendol only. Tap the chip, then add a filter.

Metrics in this data (tap to add):

Time 0 is when the data starts. The query looks back from the eval time.

Watch

Rule names follow level:metric:operations. Good name? Swipe right for good.

Swipe right for good, left for needs work. Or use the buttons.

1 / 4stall:kedai_orders:rate5m

Check

Check rule files before you load them. Real error first: tap the line that says what is wrong.

Tap the line that names the problem.

0 of 1 found

Predict

promtool says SUCCESS: 1 rules found. When does the new series appear?

You did it!

You can now save a busy query as a recording rule and check it.

Go deeper (optional)

Where the story breaks: a clerk writes a counter down once. Prometheus copies every number every 15 s, changed or not. It keeps them 15 days by default, then throws them away. And a real counter never resets; an app’s does.

Real rep: save rules.yml, then add the lines below to your other two files.

# rules.yml
groups:
  - name: kedai
    rules:
      - record: instance:node_cpu_busy:ratio_rate1m
        expr: 1 - avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[1m]))
# prometheus.yml: add at the top level, after global
rule_files:
  - rules.yml
# compose.yaml: add under the prometheus volumes
      - ./rules.yml:/etc/prometheus/rules.yml:ro

Real mission

  2. docker compose up -d
  3. docker compose exec prometheus promtool check rules /etc/prometheus/rules.yml
  5. docker compose down

0 of 5 steps done · Your ticks are saved.