Uploaded image for project: 'OpenNMS'
  1. OpenNMS
  2. NMS-13131

Nephron: Use unaligned windows for different exporters

    XMLWordPrintable

Details

    • Enhancement
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Fixed
    • None
    • 28.0.0
    • None
    • Security Level: Default (Default Security Scheme)
    • None
    • Horizon 2021 - Feb 3 - Feb 17, Horizon 2021 - Feb 17 - Mar 3, Horizon 2021 - Mar 3 - Mar 17

    Description

      ATM all windows for the different collections "fire" at the same milliseconds. This results in periodic load bursts.

      Quote from "Streaming Systems" (O'Reilly 2018): "In circumstances for which comparing across windows is unnecessary, it’s often more desirable to spread window completion load out evenly across time. This makes system load more predictable, which can reduce the provisioning requirements for handling peak load."

      In the book there is an example where a field value is used via its hashCode to calculate window offsets. In our case we could offset windows depending on the exporter node id. The resulting flow summaries would no longer all be aligned to the same window boundaries. They would also be spread over time. The ES drift plugin copes with that situation.

      We already use the FlowWindows.FlowWindow class that distinguishes between Windows of different exporters based on exporter node ids. We can use standard IntervalWindow that are offset based on the hash of node ids instead.

      Attachments

        Activity

          People

            swachter Stefan Wachter
            swachter Stefan Wachter
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.