Monitor linux server with Prometheus node exporter

by Daniel Pham
Published: Updated:
This entry is part 11 of 14 in the series Install Prometheus and Grafana on Ubuntu 18

In this article, we will monitor linux server with Prometheus node exporter. If you have installed blackbox exporter before, this article will be a little different.

With blackbox exporter, you can install it right on the prometheus server. But with node exporter, you must install it on the Linux server that needs monitoring. The node exporter here is like an agent monitor.

Install Prometheus node exporter

First, check the latest version of the node exporter at this link. At the time of writing, the latest version is 1.0.1.

We download and unzip the installation package.

wget https://github.com/prometheus/node_exporter/releases/download/v1.0.1/node_exporter-1.0.1.linux-amd64.tar.gz
tar -xzf node_exporter-1.0.1.linux-amd64.tar.gz && rm -f node_exporter-1.0.1.linux-amd64.tar.gz
cd node_exporter-1.0.1.linux-amd64

Move the binary executable file to the local bin directory.

mv node_exporter /usr/local/bin/
cd .. && rm -rf node_exporter-1.0.1.linux-amd64

Create a user for the node exporter service.

useradd -rs /bin/false node_exporter
chown node_exporter:node_exporter /usr/local/bin/node_exporter

Create a systemd file for the node exporter service.

nano /etc/systemd/system/node_exporter.service

Copy the content below into the file.

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=node_exporter
Group=node_exporter
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/node_exporter

SyslogIdentifier=node_exporter
Restart=always

[Install]
WantedBy=default.target

Reload the daemon and start the node exporter service.

systemctl daemon-reload
systemctl start node_exporter
systemctl enable node_exporter

After starting the node_exporter service, it will run on port 9100.

For those who are lazy, you can download the node exporter installation script I wrote for quick installation.

Set up Prometheus configuration

Now, we will set up prometheus to call node_exporter targets on Linux servers.

Open the prometheus configuration file.

nano /etc/prometheus/prometheus.yml

You add the following paragraph to the file.

  - job_name: 'node_exporter'
    scrape_interval: 5s
    static_configs:
    - targets:
      - hostname-linux:9100

You replace hostname-linux with the IP address or DNS name of the Linux server you need to monitor.

Next, find the section below and add the rule file path for the linux server.

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"
    - "/etc/prometheus/rules/linux.yml"

Warning: You need to open the firewall to allow the prometheus server to access port 9100 on the linux server that needs monitoring.

Set up alert rule in Prometheus

If you don’t know how to set up rules, you should refer to the article about your blackbox exporter.

Recommended Reading: Install Alertmanager and set up Prometheus alerts

You create a rule file containing the necessary warnings for the Linux server.

nano /etc/prometheus/rules/linux.yml

Then you create rules, for example an alert as below. This alert will determine that the node_exporter service on the target linux server is not working. Thereby, alert sends us a notification that the host is down.

You can download the complete rule file for Linux server at this gitlab link.

groups:
- name: Linux Server
  rules:
  - alert: Server Linux Down
    expr: up{job="node_exporter"} == 0
    for: 10s
    labels:
      severity: "Critical"
    annotations:
      Summary: 'Server "{{ $labels.instance }}" down.'

After setting up prometheus, you can reload the service.

systemctl reload prometheus

Set up the Grafana dashboard

My Grafana dashboard is using version ID 11074. You can import ID directly from Grafana.

Monitor linux server with Prometheus node exporter
Dashboard monitor linux server with prometheus node exporter.

Or download and import json from the dashboard you saved.

Conclusion

So you have completed the steps to set up a Linux monitor server with Prometheus node exporter. This tool is quite effective, it pushes almost complete Linux resource server information.

5 1 vote
Article Rating
Series Navigation«« Previous part: How to delete old targets in Prometheus databaseNext part: Monitor windows server with Prometheus wmi_exporter »»

You may also like

Subscribe
Notify of
guest
1 Comment
Newest
Oldest Most Voted
Inline Feedbacks
View all comments
truong hong minh
truong hong minh
Guest
3 years ago

thank bạn nhiều tìm nhiều nguồn lắm mới ra được bạn

DevOps Lite is a personal blog specializing in technology with main topics about DevOps, DevSecOps, SRE and System Administrator. Articles are shared for free and contributed to the community.

SUPPORT US

FOLLOW US

Subscribe my Newsletter for new blog posts. Stay updated from your inbox!

© 2021-2024 DevOpsLite.com – All rights reserved.

Please write sources “DevOpsLite.com” when using articles from this website.

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More

2
0
Would love your thoughts, please comment.x
()
x

Adblock Detected

Please support us by disabling your AdBlocker extension from your browsers for our website.