Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Develop Timeseries Integration Layer

Description

Original focus of ticket:
Our APIs for persisting and retrieving time series metrics were originally designed and optimized for rrdtool and variants. We did a number of updates when adding support for Newts, but they are still very RRD centric.

Since that time, there has been a great increase in the amount of time series databases and persistence solutions available. To name a few:

  • Elasticsearch

  • InfluxDB

  • TimescaleDB

  • Cortex (Prometheus)

Within the scope of this issue we should review the existing APIs, document their functionality and propose enhancements that would make it easier / possible to integrate with additional time series databases while being able to leverage the features they provide.

extended focus

After reviewing the current timeseries implementation we extended the focus of the ticket and decided to add an integration layer for timeseries databases to OpenNMS. The integration layer allows for the easy integration of timeseries databases without knowledge of the inner workings of OpenNMS. The implementation of a simple interface is sufficient:
https://github.com/OpenNMS/opennms-integration-api/blob/master/api/src/main/java/org/opennms/integration/api/v1/timeseries/TimeSeriesStorage.java

New database integrations can be added via OSGI Plugin. Examples:

The integration layer can be activated via opennms.properties:

Additionally to that an actual implementation of needs to be started via karaf shell. See the description in the above mentioned plugins for more information.

Pull Requests:
OpenNMS:

Integration API:

Acceptance / Success Criteria

None

Lucidchart Diagrams

Activity

Show:

Jesse White March 18, 2020 at 2:19 PM

Sean Torres December 6, 2019 at 12:28 PM

Probably also a good time to see about https://www.m3db.io/ as a potential DB

Patrick Schweizer November 6, 2019 at 8:01 PM

Working document for the documentation of time series persistence API. I used a Google doc for better collaboration, e can later copy the content somewhere else:
https://docs.google.com/document/d/1jHWYp9uBUAB7SrZL_z1OmyQuwAG73KLu2DyeLG0fYMM/edit?usp=sharing

Patrick Schweizer November 6, 2019 at 5:55 PM

The 3 entrypoints to time series data are:

  • writing: org.opennms.netmgt.collection.api.Persister

  • availability of metrics: org.opennms.netmgt.dao.api.ResourceStorageDao

  • retrieving from db: org.opennms.netmgt.measurements.api.MeasurementFetchStrategy

Fixed

Details

Assignee

Reporter

Sprint

Fix versions

Priority

PagerDuty

Created November 6, 2019 at 2:59 PM
Updated March 24, 2020 at 8:16 PM
Resolved March 24, 2020 at 8:16 PM