Uploaded image for project: 'OpenNMS'
  1. OpenNMS
  2. NMS-13453

Nephron: CortexIo GC global state

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: Next
    • Fix Version/s: Nephron-0.2.2
    • Component/s: None
    • Security Level: Default (Default Security Scheme)
    • Labels:
      None
    • Sprint:
      Horizon 2021 - Sep 1 - 15
    • HB Backlog Status:
      Backlog NG

      Description

      CortexIo uses a stateful transform and the global window to attach index number to samples to meet Cortex's time ordering constraint.

      For each watched CompoundKey (exporter/interface/tos/application, ...) a list of so called "flushed" timestamps is kept in state that. The list of timestamps for a particular key should be removed from Beam's state after the allowedLateness for a key has passed without seeing a flow for that key for the maximum flushed timestamp.

      A possible solution could be based on Beam's timer mechanism using first a timer in the event time domain (waiting for the watermark to pass the on time condition) and then another timer in the processing time domain for clearing the state "max allowed lateness" has passed.

      While certainly doable, I asked on the Beam mailing list if there is a simpler solution to that problem (cf. GC state in case of global window).

       

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              swachter Stefan Wachter
              Reporter:
              swachter Stefan Wachter
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration