Jexl Measurements API enhancements

Description

Two enhancements to measurements api to allow jexl formulae to be referenced as constants and more complex calculations which use previous samples.

A/ stored formulae

The current use case for jexl expressions expects the jexl formulae must be explicitly supplied in the query. However in our use case, we want to be able to store mathematical formulae and the constants used in formulae as string constants stored alongside  the raw measurement data. This will allow us to create formulae for each measurement device which apply data corrections to raw measurements.

Use case: Cheap Iot environment sensors are inherently inaccurate and have non-linear response curves. However their measurements  can be calibrated against accurate devices by applying a correction formulae to the raw retrieved data. However each device must needs a different formulae and so saving that formulae as a constant will make accurate data retrieval much easier to achieve.

B/ reference previous values in time series

The measurements API allows jexl expressions which can cross reference other measurements but not previous measurements in the time series.More complex data processing such as finite impulse response filters (FIR) require the ability to reference previous samples this enabling formulae such as y = a*fx( n ) + b*fx(n-1)+ c*fx(n-2)etc

 

Acceptance / Success Criteria

None

Lucidchart Diagrams

Activity

Show:

Craig Gallen November 28, 2017 at 4:46 PM

incorporated changes following  Jessi's comments on previous pull request on new branch NMS-9743-v2 https://github.com/OpenNMS/opennms/tree/jira/NMS-9743-v2

which targets origin/release-21.0.2

and bamboo

https://bamboo.opennms.org/browse/OPENNMS-ONMS1940

Craig Gallen November 27, 2017 at 11:25 AM

https://github.com/OpenNMS/opennms/tree/jira/NMS-9743

Completed changes and added documentation to measurements api.

See pull request https://github.com/OpenNMS/opennms/pull/1770

 

 

 

Craig Gallen November 20, 2017 at 1:43 PM

Note: currently it is possible to create a jexl expression which instantiates a new jexel engine and evaluates a formula which is suppliied as a string. For a  line formula we can use a constant formulae= "m*x+y". However the context has to be recreated explicitly for the expression engine.

new("org.apache.commons.jexl2.JexlEngine").createExpression(formulae).evaluate(new("org.apache.commons.jexl2.MapContext","".class.forName("com.google.common.collect.ImmutableMap").of("m", 1.5, "x", PM1, "y", -10)))

If thecontext was directly injected as a self reference __context in the context, this simplifies to 3 string constants and an expression;

formulae= "m*x+y"

m=1.5

y=-10

expression becomes new("org.apache.commons.jexl2.JexlEngine").createExpression(formulae).evaluate(__context)))

 

Fixed

Details

Assignee

Reporter

Components

Fix versions

Priority

PagerDuty

Created November 20, 2017 at 1:30 PM
Updated December 1, 2017 at 1:44 AM
Resolved December 1, 2017 at 1:44 AM