Uploaded image for project: 'OpenNMS'
  1. OpenNMS
  2. NMS-13535

Nephron: Support running on Flink 1.13

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Nephron-0.2.2
    • Component/s: None
    • Security Level: Default (Default Security Scheme)
    • Labels:
      None
    • Sprint:
      Horizon 2021 - Sep 1 - 15

      Description

      ATM Nephron can not be run on Flink 1.13. The problem manifests itself by the exception shown below.

      The reason is that somehow the artifact com.fasterxml.jackson.module:jackson-module-jaxb-annotations:2.3.1 makes it into the classpath. That artifact is rather old (from 2013) and no more compatible with newer versions of Jackson.

      (The artifact can be found in the /tmp/blobStore-?/job_?/ folder where flink seems to unpack data for deployed jobs. The artifact is named blob_p-??? but looking into it one can find its manifest file.)

      A workaround is to add a dependency of that artifact to the Nephron's main module that is compatible with the used jackson version. For example: com.fasterxml.jackson.module:jackson-module-jaxb-annotations:2.12.3. This results in both artifacts being in the blobStore/job folder. Luckily the newer version gets picked.

      TODO: Investigate where the dependency on the old version comes from and remove that dependency.
       

      Caused by: java.lang.NoSuchMethodError: 'java.lang.String com.fasterxml.jackson.databind.util.BeanUtil.okNameForGetter(com.fasterxml.jackson.databind.introspect.AnnotatedMethod)'
              at com.fasterxml.jackson.module.jaxb.JaxbAnnotationIntrospector.findNameForSerialization(JaxbAnnotationIntrospector.java:669) ~[?:?]
              at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.findNameForSerialization(AnnotationIntrospectorPair.java:596) ~[?:?]
              at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector._addGetterMethod(POJOPropertiesCollector.java:742) ~[?:?]
              at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector._addMethods(POJOPropertiesCollector.java:687) ~[?:?]
              at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.collectAll(POJOPropertiesCollector.java:422) ~[?:?]
              at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.getJsonValueAccessor(POJOPropertiesCollector.java:270) ~[?:?]
              at com.fasterxml.jackson.databind.introspect.BasicBeanDescription.findJsonValueAccessor(BasicBeanDescription.java:258) ~[?:?]
              at com.fasterxml.jackson.databind.ser.BasicSerializerFactory.findSerializerByAnnotations(BasicSerializerFactory.java:391) ~[?:?]
              at com.fasterxml.jackson.databind.ser.BeanSerializerFactory._createSerializer2(BeanSerializerFactory.java:220) ~[?:?]
              at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:169) ~[?:?]
              at com.fasterxml.jackson.databind.SerializerProvider._createUntypedSerializer(SerializerProvider.java:1473) ~[?:?]
              at com.fasterxml.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1421) ~[?:?]
              at com.fasterxml.jackson.databind.SerializerProvider.findValueSerializer(SerializerProvider.java:520) ~[?:?]
              at com.fasterxml.jackson.databind.SerializerProvider.findTypedValueSerializer(SerializerProvider.java:798) ~[?:?]
              at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:308) ~[?:?]
              at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4292) ~[?:?]
              at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:4245) ~[?:?]
              at org.apache.beam.sdk.options.ProxyInvocationHandler$Serializer.serialize(ProxyInvocationHandler.java:679) ~[?:?]
              at org.apache.beam.sdk.options.ProxyInvocationHandler$Serializer.serialize(ProxyInvocationHandler.java:648) ~[?:?]
              at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[?:?]
              at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[?:?]
              at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4487) ~[?:?]
              at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3742) ~[?:?]
              at org.apache.beam.runners.core.construction.SerializablePipelineOptions.serializeToJson(SerializablePipelineOptions.java:68) ~[?:?]
              at org.apache.beam.runners.core.construction.SerializablePipelineOptions.<init>(SerializablePipelineOptions.java:44) ~[?:?]
              at org.apache.beam.runners.flink.translation.types.CoderTypeInformation.<init>(CoderTypeInformation.java:48) ~[?:?]
              at org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper.initializeState(UnboundedSourceWrapper.java:438) ~[?:?]
              at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.tryRestoreFunction(StreamingFunctionUtils.java:189) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.restoreFunctionState(StreamingFunctionUtils.java:171) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.initializeState(AbstractUdfStreamOperator.java:96) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.initializeOperatorState(StreamOperatorStateHandler.java:118) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:290) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:441) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:582) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.call(StreamTaskActionExecutor.java:100) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.streaming.runtime.tasks.StreamTask.executeRestore(StreamTask.java:562) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:647) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:537) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:759) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
              at java.lang.Thread.run(Thread.java:834) ~[?:?]
       

       

       

       

       

        Attachments

          Activity

            People

            Assignee:
            swachter Stefan Wachter
            Reporter:
            swachter Stefan Wachter
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Git Integration