Uploaded image for project: 'OpenNMS'
  1. OpenNMS
  2. NMS-12580

Improve node cache in flow document enrichment



    • Story
    • Status: Resolved (View Workflow)
    • Major
    • Resolution: Fixed
    • None
    • 26.0.0
    • None
    • Security Level: Default (Default Security Scheme)
    • None


      In the flow DocumentEnricher, we current enrich the documents with node information related to the export and src/dst addresses of the flow: https://github.com/OpenNMS/opennms/blob/92e4031c75cbfff6c41e42dabe074ba362e9528e/features/flows/elastic/src/main/java/org/opennms/netmgt/flows/elastic/DocumentEnricher.java#L111

      For exporters, we expect to find a matching node in most if not all cases.
      For src/dst addresses, we do not expect to find a node in most cases.

      Exporters can be expected to have a cardinality of about 1000, whereas src/dst addresses are expected to have a much higher cardinality in the order or hundreds of thousands.

      With the current structure, this would mean that we need a cache sized with cardinality(exporters) + cardinality(src&dst) addresses in order to avoid constant evictions.

      As an alternate scheme, I propose the following:

      • Always query the InterfaceToNodeCache to retrieve the node id
        • The complete set of interface to node mappings is already in memory, so this will not require any DB hits
      • Maintain a cache from node id to NodeDocument that is only used when an node id is found from the previous lookup

      Now, we only need to maintain a cache that is in the order of nodes in the system, rather than unique src/dst addresses.




            cgorantla Chandra Gorantla
            j-white Jesse White
            0 Vote for this issue
            2 Start watching this issue