Merged
Details
Assignee
Freddy ChuFreddy ChuReporter
Alejandro GalueAlejandro GalueHB Grooming Date
Apr 26, 2022HB Backlog Status
BacklogStory Points
5Components
Sprint
NoneFix versions
Affects versions
Priority
Medium
Details
Details
Assignee
Freddy Chu
Freddy ChuReporter
Alejandro Galue
Alejandro GalueHB Grooming Date
Apr 26, 2022
HB Backlog Status
Backlog
Story Points
5
Components
Sprint
None
Fix versions
Affects versions
Priority
PagerDuty
PagerDuty
PagerDuty
Created April 21, 2022 at 11:56 AM
Updated June 7, 2022 at 5:35 PM
Resolved June 3, 2022 at 1:11 PM
The Karaf command
opennms:stress-metrics
is a valuable tool to generate fake metrics to stress the persistence layer. Currently, it creates counterfeit nodes/resources fixedly, for instance:For the metric name:
metric_X_Y
X is the numeric group ID, and Y is the numeric metric ID.
For the resource IDs:
snmp/fs/STRESS/XX/tapYY/groupZZ
XX is the numeric Node ID, YY is the numeric Interface ID, and ZZ is the numeric group ID.
For specific Timeseries databases that we could use to persist metrics like Cortex, we need to understand Cardinality, and that comes into two forms:
1) Number of unique KV pairs per Series.
2) Length of the Key and the Value of each unique Pair.
I'm interested in being able to control [2] more than [1], although I provide ideas for both.
For the metric name, we can introduce a variable to control the length of the variable name. Let's say we pass a value of 5 for that variable, and we would get something like this:
metric_0_0_MMMMM
There are five extra characters at the end to increase the length of the metric name.
To make it more robust and closer to what OpenNMS does when collecting data, we can introduce a variance so some metric names would be shorter or more extensive than others. It can be pre-calculated the length, so a given variable will have the same length during the execution of the tool; for instance:
Similarly, we need something for the resources (different variable, of course):
snmp/fs/STRESS/1/tap10/group2_NNNNN
And if we can introduce some optional variance, it would be great.
Imagine that you want to store data in Cortex, and you're trying to estimate the amount of Memory and Disk required by the metrics. The number of labels and the length affect the Block Index (which contains the map between the tags and the time series data).
We won't get something close to what a customer would have with fixed values. Still, if we can control the length of the mandatory variables (metric name and resource ID), we can see how that affects the size of this Block Index and how much extra memory is required to manage that.
In terms of the "extra" labels, the Cortex plugin already allows you to configure that via properties file with entries like this:
Unfortunately, those are not sent to Cortex when using the stress-metrics command (only when Collectd is involved). I would consider this use case optional, but it would be helpful to comprehend what happens when adding more labels per each Series.
It would be wonderful to have a solution for this on
develop
for Horizon 30.