Uploaded image for project: 'OpenNMS'
  1. OpenNMS
  2. NMS-13017

When using a custom prefix, the Elasticsearch Forwarder for events and situation-feedback creates a wrong template.



    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 26.2.1, Meridian-2020.1.1, 27.0.0
    • Fix Version/s: 27.0.1
    • Component/s: Alarms, Events
    • Security Level: Default (Default Security Scheme)
    • Labels:
    • Sprint:
      Horizon 2020 - Nov 11-Nov 24, Horizon 2020 - Nov 24-Dec 9
    • HB Backlog Status:
      Backlog CM


      A customer heavily relies on index prefixes for all the integrations with Elasticsearch because their cluster is shared across multiple different OpenNMS environments.

      When this is the case, the template matching is incorrect, leading to something like this:

      # curl  http://localhost:9200/_cat/templates' 2>/dev/null | grep prod
      prod-eventsindextemplate       [prod-*]                         0
      prod-feedback                  [prod-*]                         0
      prod-alarms                    [prod-opennms-alarms-*]          1

      All the Elasticsearch features in OpenNMS were configured with this:


      This confuses the system, and the actual indexes could end up with the wrong template.

      The following is the only evidence found in the customer environment proving that the events forwarder is not working:

      2020-11-23T11:49:06,827 | ERROR | EventToIndex-Thread-11 | EventToIndex                     | 421 - org.opennms.features.opennms-es-rest - 26.2.1 | Bulk API action failed. Error response was: One or more of the items in the Bulk request failed, check BulkResult.getItems() for more information.

      The karaf.log* files are full of messages like this, as the environment in question processes on average over 300 events per second.

      From the initial list, only the alarms are properly defined. Although, depending on race conditions, the alarms template could end up with the events template and vice-versa, meaning all of them must be fixed.

      Here is what I would expect to see on a healthy system using a prefix:

      prod-eventsindextemplate       [prod-opennms-events-raw-*]                         0
      prod-feedback                  [prod-situation-feedback-*]                         0
      prod-alarms                    [prod-opennms-alarms-*]          1


          Issue Links



              cgorantla Chandra Gorantla
              agalue Alejandro Galue
              0 Vote for this issue
              2 Start watching this issue



                  Git Integration