Status: Resolved (View Workflow)
Affects Version/s: 23.0.0
Fix Version/s: 23.0.0
Security Level: Default (Default Security Scheme)
Environment:Running latest RPMs from the develop branch on the Core OpenNMS, and latest RPMs from the features/sentinel branch on 2 Sentinel VMs
I was trying latest development snapshot of Sentinel, from its RPMs, and its Docker image through Kubernetes:
In both cases, I saw that the solution appears to be working as expected, as I can see flow data on Elasticsearch, and Telemetryd is not handling flows on the OpenNMS Core instance; only Sentinel instances are processing flow data.
Typically, when debugging Kafka clients, I use Kafka Manager (a tool from Yahoo https://github.com/yahoo/kafka-manager), to inspect the Kafka cluster, see the amount of messages per topic, see the number of consumers currently connected and details about them, message lag, etc.
This tools shows the consumers from OpenNMS for Sink, RPC and Flows when Telemetryd is handling flows within OpenNMS. Now, when using Sentinel, the consumers for the flow topics are not shown.
Interestingly, if you manually check Zookeeper, or use the kafka-consumer-groups.sh command on any Kafka node, you can see the consumers, but after specifying additional flags; as they are not registered on the usual way.
The following command returns nothing:
But, the following returns some information:
Note that --members has been passed in order to see the data.
If you just list the consumer groups, the Sentinel group is shown (also in kafka-manager), but no client information is displayed.
I've been using Kafka Manager a lot, not only with OpenNMS, but also for personal projects, and I've never seen this behavior before with consumer groups.
I can provide more details if required, and I can even provide access to any of my lab. Please let me know which environment you want to check.