RRD-to-Newts converter only handles AVERAGE RRAs

Description

Importantly, the system where we're seeing this problem is collecting on a 15-minute interval rather than the default 5-minute one. It's also worth mentioning that we had to do some custom scripting to convert the RRD repository to store-by-foreign-source.

After swinging this system's persisting to Newts, tripling the default heartbeat setting for Newts fetches, and running the RRD-to-Newts converter, we expected to see resource graphs of historical time-series data gradually fill in as the converter ran, eventually meeting up with the ongoing collection. In reality, the historical data never became visible in resource graphs, despite the data from ongoing collection being visible there.

The customer took the initiative to start doing some queries directly against the Cassandra cluster, and found that historical samples were being inserted on 60-minute boundaries rather than 15-minute ones.

Environment

ems15: CentOS 7, long-running 15-minute RRD collections being converted to Newts

Acceptance / Success Criteria

None

Lucidchart Diagrams

Activity

Show:

fooker May 27, 2020 at 10:17 PM

fooker May 27, 2020 at 12:13 PM
Edited

The converter currently uses only RRAs with a correlation-function of AVERAGE. This specific RRD definition looks like the following:

The "finest" AVERAGE RRA has a step size of 4 which correlates 4 samples into one row leading to a row timing of 4*900=3600 seconds.

We can update the logic to look also include the "biggest" with a step size of 1 regardless of the Correlation Function while generating the samples.

Fixed

Details

Assignee

Reporter

Components

Sprint

Affects versions

Priority

PagerDuty

Created May 19, 2020 at 9:26 PM
Updated March 11, 2021 at 3:20 AM
Resolved June 4, 2020 at 8:11 PM