Be able to control label sizes for the stress-metrics command

Description

The Karaf command opennms:stress-metrics is a valuable tool to generate fake metrics to stress the persistence layer. Currently, it creates counterfeit nodes/resources fixedly, for instance:

For the metric name: metric_X_Y

X is the numeric group ID, and Y is the numeric metric ID.

For the resource IDs: snmp/fs/STRESS/XX/tapYY/groupZZ

XX is the numeric Node ID, YY is the numeric Interface ID, and ZZ is the numeric group ID.

For specific Timeseries databases that we could use to persist metrics like Cortex, we need to understand Cardinality, and that comes into two forms:

1) Number of unique KV pairs per Series.
2) Length of the Key and the Value of each unique Pair.

I'm interested in being able to control [2] more than [1], although I provide ideas for both.

For the metric name, we can introduce a variable to control the length of the variable name. Let's say we pass a value of 5 for that variable, and we would get something like this:

metric_0_0_MMMMM

There are five extra characters at the end to increase the length of the metric name.

To make it more robust and closer to what OpenNMS does when collecting data, we can introduce a variance so some metric names would be shorter or more extensive than others. It can be pre-calculated the length, so a given variable will have the same length during the execution of the tool; for instance:

Similarly, we need something for the resources (different variable, of course):

snmp/fs/STRESS/1/tap10/group2_NNNNN

And if we can introduce some optional variance, it would be great.

Imagine that you want to store data in Cortex, and you're trying to estimate the amount of Memory and Disk required by the metrics. The number of labels and the length affect the Block Index (which contains the map between the tags and the time series data).

We won't get something close to what a customer would have with fixed values. Still, if we can control the length of the mandatory variables (metric name and resource ID), we can see how that affects the size of this Block Index and how much extra memory is required to manage that.

In terms of the "extra" labels, the Cortex plugin already allows you to configure that via properties file with entries like this:

Unfortunately, those are not sent to Cortex when using the stress-metrics command (only when Collectd is involved). I would consider this use case optional, but it would be helpful to comprehend what happens when adding more labels per each Series.

It would be wonderful to have a solution for this on develop for Horizon 30.

Acceptance / Success Criteria

None

Lucidchart Diagrams

Activity

Show:

Freddy Chu May 27, 2022 at 8:29 PM
Edited

For the metrics & resources length. 4 options are added.

example output in gateway:

gateway         | 2022-05-27T20:26:19.620Z debug server/server.go:413 processing sample {"metric_key": "name=metric_0_0b41af1fa5218aa3118f4ffa2e0e6c4_resourceId=snmp/9/tap0/group0_f4fd7231ec2a55d1e6b8333208cc46e215c83277f4fd7231ec2a55d1e6b"}

For tagging. Stress command actually support tagging. The main reason that stress command didn't send those tags is due to your database don't contain those data in node / interface table. If you create all info needed it can sent actually. 

Merged

Details

Assignee

Reporter

HB Grooming Date

HB Backlog Status

Story Points

Components

Sprint

Affects versions

Priority

PagerDuty

Created April 21, 2022 at 11:56 AM
Updated June 7, 2022 at 5:35 PM
Resolved June 3, 2022 at 1:11 PM