World O1: The Control Room

Dashboards or alerts?

You will learn: what belongs on a screen, what deserves to wake someone up, and how to tell them apart.

Checked with Prometheus 3 on 1 Nov 2026

The problem

3 a.m. The disk is 71% full and fills slowly. Wake someone up?

Watch

A waits until you look. An alert calls a person, now. Sort each one.

Drag a card into a box. Or tap the card, then tap the box.

Try it

Build one that calls a person only when customers hurt. Pick the blocks, then test.

Build an alert rule from the blocks. The timeline shows when it would wait and when it would fire.

When (expr)

Lots of ordersOver 2% of checkouts fail

Wait first (for)

no wait5m

Your rule: rules.yml

1groups:
2- name: kedai
3rules:
4- alert: CheckoutFailing
5expr: kedai_orders_per_minute > 30

The value the alert watches

Graph as a table

Graph as a table
Time	`kedai_orders_per_minute`
0s	20
1m	21
2m	22
3m	23
4m	24
5m	25
6m	26
7m	27
8m	28
9m	29
10m	30
11m	31
12m	32
13m	33
14m	34
15m	35
16m	36
17m	37
18m	38
19m	39
20m	40
21m	39
22m	38
23m	37
24m	36
25m	35
26m	34
27m	33
28m	32
29m	31
30m	30

What the alert does over time

Grey: quiet. Amber: pending (true, but waiting). Red: firing.

{}
🔔 firing 11m to 29m

Look at time: 25m

At 25m:

FIRING CheckoutFailing

Timeline as a table

Time	`{}`
0s	quiet
1m	quiet
2m	quiet
3m	quiet
4m	quiet
5m	quiet
6m	quiet
7m	quiet
8m	quiet
9m	quiet
10m	quiet
11m	🔔 firing
12m	🔔 firing
13m	🔔 firing
14m	🔔 firing
15m	🔔 firing
16m	🔔 firing
17m	🔔 firing
18m	🔔 firing
19m	🔔 firing
20m	🔔 firing
21m	🔔 firing
22m	🔔 firing
23m	🔔 firing
24m	🔔 firing
25m	🔔 firing
26m	🔔 firing
27m	🔔 firing
28m	🔔 firing
29m	🔔 firing
30m	quiet

Predict

Your alert waits 5 minutes before it calls (for: 5m). Why wait?

Check

Is this a good alert? Swipe right for good.

Swipe right for good, left for needs work. Or use the buttons.

1 / 5

Checkout errors above 2% for 5 minutes

You did it!

You can now choose between a dashboard and an alert.

Go deeper (optional)

A good alert is urgent, needs a human, and needs it now. Everything else goes on a dashboard, a ticket or a chat message. World O3 builds real alert rules and routes them with Alertmanager.

Real mission

List the alerts that reached you last month, or imagine five for your app.
For each one, ask: did someone have to act right away?
Move every “no” to a dashboard or a ticket.

0 of 3 steps done · Your ticks are saved.

Time	`kedai_orders_per_minute`
0s	20
1m	21
2m	22
3m	23
4m	24
5m	25
6m	26
7m	27
8m	28
9m	29
10m	30
11m	31
12m	32
13m	33
14m	34
15m	35
16m	36
17m	37
18m	38
19m	39
20m	40
21m	39
22m	38
23m	37
24m	36
25m	35
26m	34
27m	33
28m	32
29m	31
30m	30

Time	`kedai_orders_per_minute`
0s	20
1m	21
2m	22
3m	23
4m	24
5m	25
6m	26
7m	27
8m	28
9m	29
10m	30
11m	31
12m	32
13m	33
14m	34
15m	35
16m	36
17m	37
18m	38
19m	39
20m	40
21m	39
22m	38
23m	37
24m	36
25m	35
26m	34
27m	33
28m	32
29m	31
30m	30

Time	`kedai_orders_per_minute`
0s	20
1m	21
2m	22
3m	23
4m	24
5m	25
6m	26
7m	27
8m	28
9m	29
10m	30
11m	31
12m	32
13m	33
14m	34
15m	35
16m	36
17m	37
18m	38
19m	39
20m	40
21m	39
22m	38
23m	37
24m	36
25m	35
26m	34
27m	33
28m	32
29m	31
30m	30