RIFT.ware Logging

The RIFT.ware logging subsystem provides a framework for the collection and distribution of event logs. In a large system with massive amounts of logs generated, RIFT.ware logging provides a way to efficiently capture and store logs. This data is made available for syslogs to provide information about the operational health of the system, as well as various aspects of applications running on the RIFT.ware platform.

The RIFT.ware platform can potentially generate large log volumes from the collection of tasklets running in each virtual machine (VM). RWLogD is a new tasklet which aggregates the logs from all the source tasklets in a VM and send it to the registered sinks. Each VM runs one instance of RWLogD tasklet. RWLogD acts as a local aggregator for a VM receiving log events and distributing them to the registered sinks.

The RIFT.ware logging subsystem provides the following functionality:

  • Carefully-designed logging framework for minimal performance affect during log generation on data path.

  • Support multiple logging sink, such as syslog, console, CLI, file, and HDFS.

  • Well-defined, thread-safe APIs for RIFT.ware tasklets to generate logs and send events.

  • Ability to generate logs from C/C++ and Python.

  • Support category for log generation to allow each module to define its own category and log events.

  • Use of Yang Notifications to define log and use auto-generated protbuf message during log generation and propagation. This allows logs to be semi-structured and available for analytics.

  • Ability to indicate logs as critical-info so that logs are generated, regardless of category/severity settings. This option is useful for bootstrap logs.

  • Plugin API-based architecture to support adding external log sinks apart from those supported by RIFT.ware.

  • Ability to write binary event logs to file in PCAP format.

  • Collect/record initial boot time log events, even when the log infrastructure is still coming up and later pushing it to RWLogD subsystem.

  • Ability to record new sessions until failure for N initial seconds, but to discard recording of successful sessions.

  • Ability to add dynamic categories and support logging filters for same to integrate with RIFT.ware logging subsystem.

  • Support to shard logs based on callid across RwLogD instance for faster lookup.

  • Support to buffer recent logs and view the same using CLI/NetConf/RestConf.

  • Support to rate limit event logs generated at source and suppress duplicate logs.