Centralising monitoring
The monitoring server and its probes: collection by polling or by pushing, SNMP, the four states of a service and confirmation, dependencies between hosts, escalation, Prometheus and its exporters, and who watches the monitor.
- 34 cards
- 5 topics
- Free
The outline, topic by topic
Each topic is weighted the way the exam weighs it.
- Concepts
- Questions
- On the diagram
- Scenarios
- Understand
A few cards from the deck
Probe: what is it?
Show the answer
An automatic test that queries a host or a service and returns a state: OK, warning, critical or unknown.
A probe is only as good as what it checks: a test that the process exists says nothing about the page the user sees.
Nagios allows 3 attempts. A probe returns OK, then critical, then critical. What is the service's state?
- Soft critical (2/3), no notification
- Hard critical, the notification goes out
- Warning, until the third attempt has happened
- OK, since there are not yet three failures
Show the answer
Soft critical (2/3), no notification
The service is already critical, but the state is only confirmed at the third attempt in a row. Until then it is soft: it shows, but nobody is notified.
To avoid false alarms, a critical result first stays … until the maximum number of attempts; it then becomes … and the notification goes out.
Show the answer
To avoid false alarms, a critical result first stays soft until the maximum number of attempts; it then becomes hard and the notification goes out.
A lost packet or a one-second load spike then wakes nobody up. The price is a delay: after the first failure, it takes another (number of attempts − 1) × retry interval before you are told.
Monitoring agent: what is it?
Show the answer
A small program installed on the monitored machine, which measures locally (processor, disk, processes) and sends the results to the monitoring server.
It sees what an outside probe cannot, such as a nearly full disk, but it is one more piece of software to install and update on every machine.
All the monitoring lights are green while the site no longer responds. What is the most likely cause?
- The probe checks the process exists, not that a page shows
- The alert threshold is set too low and drowns real alerts
- The maximum number of attempts is too low to confirm the state
- The SNMP community changed and the probes return "unknown"
Show the answer
The probe checks the process exists, not that a page shows
A process can be running, stuck, serving nobody. A useful probe imitates the user: it loads the page and checks its content.