Projects

Network Monitoring & Report Automation

Production MonitoringGrafanaPlaywrightNOC

From scripted per-area screenshots to monitoring inside eBilling: upstream and router traffic, ping, OLT optical levels, worker health and a CPU-steal-aware job gate.

Problem

NOC needed fast per-area status (down/up, excluding isolated customers). The first flow was manual: open the dashboard, filter, screenshot, send to the group — about 10 minutes each time. Later the app kept its own hand-maintained target list that duplicated Grafana, and its heavy jobs slowed the server at the worst moments.

My role

Built the first report pipeline myself, then planned the move to Prometheus/Grafana-backed monitoring inside eBilling and reviewed the implementation.

Scale

  • About 30 monitoring areas
  • 10+ routers and several upstream links graphed
  • 1,000+ ONUs with optical levels tracked

What I built

  • Stage 1: Python + Playwright pipeline that logs in, scrapes per-area tables, drops isolated rows, renders an HTML report to PNG and sends it to WhatsApp/Telegram — on a schedule or on request (monitoring <area>).
  • Stage 2: Prometheus with MikroTik and blackbox exporters, Grafana dashboards and a PPPoE exporter covering all active routers.
  • A Network tab in eBilling: upstream panel (now / peak / p95 / capacity), router table with ping, PPPoE count, traffic, CPU, memory and uptime, with stale-data badges.
  • OLT optical levels collected on a schedule; weak-signal ONUs listed for maintenance.
  • Worker heartbeat: a host script checks every queue worker and reports when one stops.
  • Admission gate: monitoring jobs wait when critical queues are busy or the VM's CPU steal is high, so installs and payments always go first.
  • Retired the hand-maintained target list once the same checks lived in the blackbox exporter (backup kept).

Architecture

  1. Devices

    • MikroTik routers
    • OLTs

  2. Collect

    • Exporters (MikroTik, blackbox, PPPoE)
    • eBilling collectors + gate

  3. Store

    • Prometheus + Grafana

  4. Show

    • Network tab
    • PNG report → WhatsApp / Telegram

Outcome

  • A per-area report went from about 10 minutes of manual work to one chat command.
  • Network status lives in the app NOC already uses, with history from Prometheus.
  • Monitoring no longer competes with installs and payments when the host is under load.
  • One source of truth for targets instead of two.

Screenshots

Demo / test data only. Real customer data never leaves production.

Network tab: routers, PPPoE, upstream usage (demo data)
Per-router traffic and top users
Per-customer 30-day usage
Same view on a phone

Network tab with routers online, PPPoE online, router traffic and upstream usage chart

Stack

PythonPlaywrightPrometheusGrafanablackbox_exporterMikroTik exporterLaravelReactRedis
GitHub

Next project

WhatsApp Ops Bot (scoped to one group)