Projects

Outage Detection & Root-Cause Hints

Shadow mode NOCGPONTR-069RCA

Collects ONU, CPE and PPPoE events, works out the likely cause (fiber cut, power off, Wi-Fi, PPPoE) and sends NOC a WhatsApp digest. Runs in shadow mode first.

Problem

The old detector only answered "is the ONU offline?", never "why". OLT and ACS data were overwritten on each poll, PPPoE was only polled, and Wi-Fi status was not real. Polling gaps looked like outages, and without a cause recorded on closed tickets, accuracy could not be measured.

My role

Defined the cause list and thresholds, planned the rollout in stages, reviewed the code, and checked the results against real tickets.

Scale

  • 1,000+ ONUs and CPEs watched
  • 3 data sources joined per customer (OLT, ACS, router)
  • About two weeks of shadow mode before any live ticket

What I built

  • One cause list used everywhere: upstream, core router, OLT down, data not readable, PON or ODP cut, drop cable LOS, power off, dead CPE, reboot loop, weak optical level, PPPoE auth/flap, Wi-Fi down, client side, unknown.
  • Event tables for ONU state changes, CPE events and PPPoE events, so there is history to look back on.
  • Cause from the OLT: LOS, dying gasp and last-down cause per ONU, with OLT clock drift corrected per session.
  • PPPoE disconnect reasons sent from the routers over syslog and mapped to customers.
  • Wi-Fi layer from GenieACS: radio status, channel, clients and signal, refreshed by a versioned ACS provision with a check and a one-command rollback.
  • Automatic diagnosis per customer and a "Check now" button that pulls fresh data on demand.
  • Shadow mode: alerts are computed and logged without creating tickets; a digest goes to the NOC group and an accuracy report compares results with real ticket causes.

Architecture

  1. Sources

    • OLT collectors (ZTE / Huawei)
    • GenieACS (TR-069)
    • MikroTik syslog (PPPoE)

  2. Store

    • Event store

  3. Logic

    • Detector + cause classifier

  4. Output

    • NOC panel + "Check now"
    • WhatsApp digest
    • Accuracy report
"Check now" refreshes OLT and ACS data for one customer on demand.

Outcome

  • NOC sees a likely cause next to each offline customer instead of a bare "down".
  • Wi-Fi complaints can be told apart from fiber problems before a technician is sent.
  • Polling gaps are labelled "data not readable" instead of raising false outages.
  • Accuracy is measured from closed tickets, so thresholds are tuned from data.

Screenshots

Demo / test data only. Real customer data never leaves production.

Deteksi Gangguan in shadow mode: likely cause, history, affected customers (demo data)
Automatic diagnosis on a ticket: cause, confidence, evidence (phone blurred)
Active outages grouped by ODP / PON
NOC view: incidents, coverage, mapping gaps
Wi-Fi layer from GenieACS (addresses blurred)
Customer detail with network enforcement state

Outage detection tab in test mode with likely cause, status, what would happen if live and affected customers

Stack

LaravelMySQLRedisZTE / Huawei OLT CLIGenieACS NBI + provisionsMikroTik syslogWhatsApp gateway

Next project

Network Monitoring & Report Automation