Panoptes: An Open Source Network Telemetry Ecosystem
Panoptes (https://github.com/yahoo/panoptes) is an open source Python-based network telemetry ecosystem. Panoptes is designed as an extensible, plugin-based platform to build network telemetry services and consists of three core frameworks: one for the discovery of resources to collect telemetry for, one for collecting ‘enrichment’ metadata, and one for collecting telemetry. Designed as a distributed and scalable system, we have worked hard to minimize single points of failure and performance bottlenecks within Panoptes by implementing clustering and/or HA/failover of all services. At Oath, we have been using Panoptes to monitor our network infrastructure, which serves content to 1 billion users and during this talk, I’ll share why we built Panoptes, how we built it, what we learned along the way, where we plan to go next, and how you can use it.