Uploaded image for project: 'OpenNMS'
  1. OpenNMS
  2. NMS-13785

Error responses are not handled correctly when handling ElasticSearch responses

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Minor
    • Resolution: Fixed
    • 29.0.1
    • 29.0.11, Meridian-2022.1.5, 30.0.1
    • None
    • Security Level: Default (Default Security Scheme)
    • None
    • Horizon - May 11 - 25, Horizon - May 26 - June 9, Horizon 22 - Jun 9 - 23
    • Reviewed

    Description

      When API calls to ElasticSearch cluster gives API response errors during flow persistence we see exceptions in our telemetryd log like this:

      java.lang.ClassCastException: class org.opennms.netmgt.flows.elastic.FlowDocument cannot be cast to class org.opennms.features.jest.client.bulk.FailedItem (org.opennms.netmgt.flows.elastic.FlowDocument is in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @71fcce9a; org.opennms.features.jest.client.bulk.FailedItem is in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @6f7b7ea1)
          at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
          at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?]
          at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
          at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
          at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
          at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
          at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
          at org.opennms.netmgt.flows.elastic.PersistenceException.getDetailedLogMessages(PersistenceException.java:54) ~[?:?]
          at org.opennms.netmgt.telemetry.protocols.flows.AbstractFlowAdapter.handleMessageLog(AbstractFlowAdapter.java:138) ~[org.opennms.features.telemetry.protocols.flows-29.0.1.jar:?]
          at org.opennms.netmgt.telemetry.daemon.TelemetryMessageConsumer.handleMessage(TelemetryMessageConsumer.java:110) ~[org.opennms.features.telemetry.daemon-29.0.1.jar:?]
          at org.opennms.netmgt.telemetry.daemon.TelemetryMessageConsumer.handleMessage(TelemetryMessageConsumer.java:58) ~[org.opennms.features.telemetry.daemon-29.0.1.jar:?]
          at org.opennms.core.ipc.sink.common.AbstractMessageConsumerManager.lambda$dispatch$0(AbstractMessageConsumerManager.java:100) ~[org.opennms.core.ipc.sink.common-29.0.1.jar:?]
          at java.lang.Iterable.forEach(Iterable.java:75) ~[?:?]
          at org.opennms.core.ipc.sink.common.AbstractMessageConsumerManager.dispatch(AbstractMessageConsumerManager.java:100) ~[org.opennms.core.ipc.sink.common-29.0.1.jar:?]
          at org.opennms.core.ipc.sink.camel.client.CamelLocalMessageDispatcherFactory.dispatch(CamelLocalMessageDispatcherFactory.java:65) ~[org.opennms.core.ipc.sink.camel.client-29.0.1.jar:?]
          at org.opennms.core.ipc.sink.camel.client.CamelLocalMessageDispatcherFactory.dispatch(CamelLocalMessageDispatcherFactory.java:53) ~[org.opennms.core.ipc.sink.camel.client-29.0.1.jar:?]
          at org.opennms.core.ipc.sink.common.AbstractMessageDispatcherFactory.timedDispatch(AbstractMessageDispatcherFactory.java:86) ~[org.opennms.core.ipc.sink.common-29.0.1.jar:?]
          at org.opennms.core.ipc.sink.common.AbstractMessageDispatcherFactory$1.dispatch(AbstractMessageDispatcherFactory.java:127) ~[org.opennms.core.ipc.sink.common-29.0.1.jar:?]
          at org.opennms.core.ipc.sink.common.AbstractMessageDispatcherFactory$1.dispatch(AbstractMessageDispatcherFactory.java:124) ~[org.opennms.core.ipc.sink.common-29.0.1.jar:?]
          at org.opennms.core.ipc.sink.aggregation.AggregatingMessageProducer.send(AggregatingMessageProducer.java:54) ~[org.opennms.core.ipc.sink.common-29.0.1.jar:?]
          at org.opennms.core.ipc.sink.common.AsyncDispatcherImpl.dispatchFromQueue(AsyncDispatcherImpl.java:128) ~[org.opennms.core.ipc.sink.common-29.0.1.jar:?]
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
          at org.opennms.core.concurrent.LogPreservingThreadFactory$1.run(LogPreservingThreadFactory.java:88) [opennms-util-29.0.1.jar:?]
          at java.lang.Thread.run(Thread.java:834) [?:?]
      

      It seems like our code can't handle the ElasticSearch API error responses and tries to parse them as flow messages. The log should give an error log entry with the error message from the failed REST API call and which endpoint was used.

      Attachments

        Issue Links

          Activity

            People

              Hutchinson James Hutchinson
              swachter Stefan Wachter
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: