When using Kafka as the implementation for the Sink API, we should automatically collect and graph metrics related to our consumers and topics.
In particular we should be monitoring:
- The lag of the OpenNMS consumer (different between the topic offsets and the consumer offsets) for all the topics
- The ingestion rate of the different topics (at what rate a messages being inserted in the topics)
The implementation must not require us to add additional data collection configurations or graphs definitions for each of the topics we monitor. New topics should be monitored automatically.