Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

All work

Select view

Select search mode

 
13 of 13

Classification Engine reload causes OOM when defining a bunch of rules

Fixed

Description

When defining a bunch of rules an engine reload may result in an OOM exception.

To reproduce the issue, simply create a rule which sets the port value to 1-65535 or omits the value completely. Try increasing the number of rules by 5 and watch the memory consumption on each reload of the engine. At some point the JVM runs out of memory. For me it was around 20-25. Please note, that with omnidirectional=true the JVM runs out of memory more quickly.

Acceptance / Success Criteria

None

Lucidchart Diagrams

Details

Assignee

Reporter

Sprint

Fix versions

Priority

PagerDuty

Created December 4, 2019 at 1:45 PM
Updated December 9, 2019 at 2:44 PM
Resolved December 9, 2019 at 2:44 PM

Activity

Markus von RüdenDecember 4, 2019 at 3:28 PM

Markus von RüdenDecember 4, 2019 at 1:53 PM
Edited

The way the classification engine works internally is, that all rules are mapped to actual ports, even if they define a range of port. The idea behind this is, that while classifying an incoming request the ports are known and the lookup is very fast (only 2 lookups -> src & dst ports).

A Rule on the other hand is converted to a list of matchers.
A rule defining a port, protocol and ip address is converted to 3 matchers, where each is determing if parts of the incoming request match:

  • is port matching

  • is protocol matching

  • is ip matching

Internally, if a rule is defining a port range, let's say 1-65535 the rule is duplicated 65535 times (resulting in a total of 65536 rules) and applied to mapped port. But with the duplication of each rule also the number of classifiers is duplicated, thus resulting in a huge increase in memory consumption.

For a rule which defines a protocol, an ip address and applies to ALL ports, the memory impact for the current implementation is roughly: 20 MB.

Please note if omnidirectional is set to true, the number of rules is doubled, resulting in doubling the memory usage for the classification rules.