Uploaded image for project: 'OpenNMS'
  1. OpenNMS
  2. NMS-12343

XMLMarshalException in ipc.log: Unexpected close tag </inst>; expected </instance>

    XMLWordPrintable

    Details

    • Sprint:
      Horizon 2019 - October 9th, Horizon 2019 - October 16th, Horizon 2019 - October 23rd

      Description

      The following exception stack trace appears repeatedly in ipc.log:

       java.lang.RuntimeException: javax.xml.bind.UnmarshalException
       - with linked exception:
      [Exception [EclipseLink-25004] (Eclipse Persistence Services - 2.5.1.v20130918-f2b9fc5): org.eclipse.persistence.exceptions.XMLMarshalException
      Exception Description: An error occurred unmarshalling the document
      Internal Exception: com.ctc.wstx.exc.WstxParsingException: Unexpected close tag </inst>; expected </instance>.
       at [row,col {unknown-source}]: [30523,28]]
      	at org.opennms.core.xml.XmlHandler.unmarshal(XmlHandler.java:87) ~[org.opennms.core.xml-25.0.0.jar:?]
      	at org.opennms.core.rpc.xml.AbstractXmlRpcModule.unmarshalResponse(AbstractXmlRpcModule.java:81) ~[org.opennms.core.ipc.rpc.xml-25.0.0.jar:?]
      	at org.opennms.core.ipc.rpc.kafka.KafkaRpcClientFactory$ResponseHandler.sendResponse(KafkaRpcClientFactory.java:383) ~[org.opennms.core.ipc.rpc.kafka-25.0.0.jar:?]
      	at org.opennms.core.ipc.rpc.kafka.KafkaRpcClientFactory$KafkaConsumerRunner.lambda$run$0(KafkaRpcClientFactory.java:488) ~[org.opennms.core.ipc.rpc.kafka-25.0.0.jar:?]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
      	at java.lang.Thread.run(Thread.java:834) [?:?]
      Caused by: javax.xml.bind.UnmarshalException
      	at org.eclipse.persistence.jaxb.JAXBUnmarshaller.handleXMLMarshalException(JAXBUnmarshaller.java:980) ~[org.eclipse.persistence.moxy-2.5.1.jar:?]
      	at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:588) ~[org.eclipse.persistence.moxy-2.5.1.jar:?]
      	at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:182) ~[org.eclipse.persistence.moxy-2.5.1.jar:?]
      	at org.opennms.core.xml.XmlHandler.unmarshal(XmlHandler.java:85) ~[org.opennms.core.xml-25.0.0.jar:?]
      	... 6 more
      Caused by: org.eclipse.persistence.exceptions.XMLMarshalException: 
      Exception Description: An error occurred unmarshalling the document
      Internal Exception: com.ctc.wstx.exc.WstxParsingException: Unexpected close tag </inst>; expected </instance>.
       at [row,col {unknown-source}]: [30523,28]
      	at org.eclipse.persistence.exceptions.XMLMarshalException.unmarshalException(XMLMarshalException.java:114) ~[org.eclipse.persistence.core-2.5.1.jar:?]
      	at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.convertSAXException(SAXUnmarshaller.java:996) ~[org.eclipse.persistence.core-2.5.1.jar:?]
      	at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:903) ~[org.eclipse.persistence.core-2.5.1.jar:?]
      	at org.eclipse.persistence.oxm.XMLUnmarshaller.unmarshal(XMLUnmarshaller.java:659) ~[org.eclipse.persistence.core-2.5.1.jar:?]
      	at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:585) ~[org.eclipse.persistence.moxy-2.5.1.jar:?]
      	at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:182) ~[org.eclipse.persistence.moxy-2.5.1.jar:?]
      	at org.opennms.core.xml.XmlHandler.unmarshal(XmlHandler.java:85) ~[org.opennms.core.xml-25.0.0.jar:?]
      	... 6 more
      Caused by: com.ctc.wstx.exc.WstxParsingException: Unexpected close tag </inst>; expected </instance>.
       at [row,col {unknown-source}]: [30523,28]
      	at com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:621) ~[woodstox-core-5.0.3.jar:5.0.3]
      	at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:491) ~[woodstox-core-5.0.3.jar:5.0.3]
      	at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:475) ~[woodstox-core-5.0.3.jar:5.0.3]
      	at com.ctc.wstx.sr.BasicStreamReader.reportWrongEndElem(BasicStreamReader.java:3365) ~[woodstox-core-5.0.3.jar:5.0.3]
      	at com.ctc.wstx.sr.BasicStreamReader.readEndElem(BasicStreamReader.java:3292) ~[woodstox-core-5.0.3.jar:5.0.3]
      	at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2911) ~[woodstox-core-5.0.3.jar:5.0.3]
      	at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1123) ~[woodstox-core-5.0.3.jar:5.0.3]
      	at org.eclipse.persistence.internal.oxm.record.XMLStreamReaderReader.parse(XMLStreamReaderReader.java:98) ~[org.eclipse.persistence.core-2.5.1.jar:?]
      	at org.eclipse.persistence.internal.oxm.record.XMLStreamReaderReader.parse(XMLStreamReaderReader.java:86) ~[org.eclipse.persistence.core-2.5.1.jar:?]
      	at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:895) ~[org.eclipse.persistence.core-2.5.1.jar:?]
      	at org.eclipse.persistence.oxm.XMLUnmarshaller.unmarshal(XMLUnmarshaller.java:659) ~[org.eclipse.persistence.core-2.5.1.jar:?]
      	at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:585) ~[org.eclipse.persistence.moxy-2.5.1.jar:?]
      	at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:182) ~[org.eclipse.persistence.moxy-2.5.1.jar:?]
      	at org.opennms.core.xml.XmlHandler.unmarshal(XmlHandler.java:85) ~[org.opennms.core.xml-25.0.0.jar:?]
      	... 6 more
      

      Indeed, on extracting the 2.1MB response document (available as bad-response-document.xml.gz on support ticket linked in Environment), the problem is plainly visible at line 30523:

            <result>
               <base>.1.3.6.1.2.1.31.1.1.1.8</base>
               <instance>844</instance>
               <value type="70">AA==</value>
            </result>
            <result>
               <base>.1.3.6.1.2.1.31.1.1.1.8</base>
               <instance>845</inst<snmp-response>
         <response correlation-id="0">
            <result>
               <base>.1.3.6.1.2.1.2.1</base>
               <instance>0</instance>
               <value type="2">AVI=</value>
            </result>
         </response>
       

      The method in play is KafkaRpcClientFactory.sendResponse.

      It looks as if the response document were truncated in mid-stream and its remainder overwritten with another document of the same schema.

      I'm awaiting verification from the reporting customer that the Minions are all versioned in lock-step with OpenNMS.

        Attachments

          Activity

            People

            • Assignee:
              cgorantla Chandra Gorantla
              Reporter:
              jeffg Jeff Gehlbach
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: