User Tools

Site Tools


tasks:grafana_agent_setup_for_lab_nodes_monitoring

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
tasks:grafana_agent_setup_for_lab_nodes_monitoring [2026/04/16 12:12] – akraitmantasks:grafana_agent_setup_for_lab_nodes_monitoring [2026/05/03 14:31] (current) – akraitman
Line 4: Line 4:
 **Grafana Agent Setup for Lab Nodes Monitoring** **Grafana Agent Setup for Lab Nodes Monitoring**
  
-### Overview+** Overview **
  
 The grafana_agent Ansible role deploys **Grafana Agent** (static mode) on Lab Nodes to collect system metrics and forward them to a central **Mimir** instance. The grafana_agent Ansible role deploys **Grafana Agent** (static mode) on Lab Nodes to collect system metrics and forward them to a central **Mimir** instance.
Line 10: Line 10:
 This setup replaces or coexists with the previous Prometheus + node_exporter approach, providing a lightweight agent that pushes metrics into Mimir for long-term storage and Grafana dashboards. This setup replaces or coexists with the previous Prometheus + node_exporter approach, providing a lightweight agent that pushes metrics into Mimir for long-term storage and Grafana dashboards.
  
-### Architecture & Data Flow+** Architecture & Data Flow **
  
  
Line 32: Line 32:
 1. **node_exporter**   1. **node_exporter**  
    - Installed on every host (prometheus-node-exporter on Debian/Ubuntu, node_exporter on RedHat via EPEL)      - Installed on every host (prometheus-node-exporter on Debian/Ubuntu, node_exporter on RedHat via EPEL)  
-   - Exposes standard host metrics (CPU, memory, disk, network, etc.) on http://localhost:9100/metrics`+   - Exposes standard host metrics (CPU, memory, disk, network, etc.) on http://localhost:9100/metrics
  
 2. **Grafana Agent**   2. **Grafana Agent**  
    - Lightweight Prometheus-compatible agent      - Lightweight Prometheus-compatible agent  
-   - Scrapes `node_exporter` locally from localhost:9100  +   - Scrapes node_exporter locally from localhost:9100  
    - Relabels the instance using the host’s FQDN      - Relabels the instance using the host’s FQDN  
    - Forwards all collected metrics to the central Mimir using the remote_write protocol    - Forwards all collected metrics to the central Mimir using the remote_write protocol
Line 44: Line 44:
    - Accepts metrics via the remote_write protocol with basic authentication    - Accepts metrics via the remote_write protocol with basic authentication
  
-### Configuration Details+** Configuration Details **
  
 **Grafana Agent config** (/etc/grafana-agent.yaml on target hosts): **Grafana Agent config** (/etc/grafana-agent.yaml on target hosts):
  
 - Global scrape interval: 60s (configurable via scrape_interval_global) - Global scrape interval: 60s (configurable via scrape_interval_global)
-- Scrapes the node job from `localhost:9100`+- Scrapes the node job from localhost:9100
 - Applies relabeling: - Applies relabeling:
-  - instance`: <fqdn>:9100 +  - instance: <fqdn>:9100 
-  - nodename`: <fqdn>+  - nodename: <fqdn>
 - Forwards metrics to Mimir with basic auth credentials stored in the secrets repository(in mimir_password.yml): - Forwards metrics to Mimir with basic auth credentials stored in the secrets repository(in mimir_password.yml):
  
-- mimir_username +** Mimir Endpoints ** 
-- mimir_password +  Internal
-- Grafana repository URLs (grafana_apt_repo_key_url, grafana_apt_repo_url, grafana_rpm_repo_url, grafana_rpm_repo_key_url) +
-- agent_mimir_url +
- +
-### Mimir Endpoint +
   https://mimir-jenkins-monitoring.apps.pok.os.sepia.ceph.com/api/v1/push   https://mimir-jenkins-monitoring.apps.pok.os.sepia.ceph.com/api/v1/push
 +  
 +  External
 +  https://mimir.ceph.com/api/v1/push
  
tasks/grafana_agent_setup_for_lab_nodes_monitoring.1776341541.txt.gz · Last modified: by akraitman