Provide a way to configure the date formats used by the Syslog NBI
Description
Acceptance / Success Criteria
Lucidchart Diagrams
Activity
Chandra Gorantla May 30, 2018 at 3:06 PM
For now, I limited this to syslog north bounder.
Alejandro Galue May 29, 2018 at 4:53 PM
You're welcome!
Alternatively, you can modify the subject of this issue to be something like "Provide a way to configure the date formats used by the Syslog and JMS NBI", and keep the current state of the PR. That way, the solution "reflects" what's on the PR, so we can always say that the the other NBIs are ourside the scope of this issue, instead of improviding an enhancement for additional NBI implementation (call it JMS) forgetting the other onces. Then, we would require a new JIRA issues, as there is no immediate need for a solution on those cases.
But I'll leave the decision to you.
Chandra Gorantla May 29, 2018 at 4:31 PM
Thanks @Alejandro Galue for the explanation. Although changes seems to be minor either way, I will restrict this to syslog.
Alejandro Galue May 29, 2018 at 4:11 PM
To simplify the work here:
Either keep only the solution for the Syslog NBI, and remove the one for JMS; or, make the solution for all the NBI implementations where I can add and use custom dates, as it is not fair to include JMS besides the required one, but exclude Email (for example).
Alejandro Galue May 29, 2018 at 4:09 PM
Here is the background for the Email NBI: https://opennms.atlassian.net/browse/HZN-565#icft=HZN-565.
Like a normal email, you can design the template used for the subject and the message you'll see on your email client. Nothing prevents you to add dates there. Also, nothing prevents you to have a software receiving and processing emails, expecting to find the date on a specific format, similar to what the customer has for Syslog (hence this issue).
For example, the following is valid and exposes the problem:
<destination>
<name>google</name> <!-- Have to match an sendmail-config's entry on javamail-configuration.xml -->
<from>opennms@domain.com</from>
<to>everybody@opennms.org</to>
<filter name="Servers">
<rule>foreignSource matches '^Servers.*'</rule>
<to>agalue@opennms.org, david@opennms.org</to>
<subject>${foreignId} : ${logMsg}</subject>
<body>ALARM ID:${alarmId} NODE:${nodeLabel} IP:${ipAddr} FIRST:${firstOccurrence} LAST:${lastOccurrence} COUNT:${count} UEI:${alarmUei} SEV:${severity} ${logMsg}</body>
</filter>
...
</destination>Note how I'm building the body of the email.
Here is the background for the SNMP Trap NBI: https://opennms.atlassian.net/browse/HZN-564#icft=HZN-564
In this particular case I agree it is harder to justify having it, and build a usable example, as I would have to review the code (as it is not as obvious like the Email counterpart).
A customer has found a discrepancy when comparing how the Syslog Northbounder behaves on Meridian 2016 (their production environment0 and latest Horizon (their development environment).
For example, let's say that the message template is configured like this:
<message-format>ALARM ID:${alarmId} NODE:${nodeLabel} IP:${ipAddr} FIRST:${firstOccurrence} LAST:${lastOccurrence} COUNT:${count} UEI:${alarmUei} SEV:${severity} ${logMsg}</message-format>On Meridian 2016, the message is rendered like this:
ALARM ID:7886101 NODE:server01 IP:10.0.0.1 FIRST:2016-11-02 08:35:29.686 LAST:2017-02-06 13:18:04.507 COUNT:85466 ...Note the format of the dates.
On latest develop (in their case the Drift Branch Build 90), the message is rendered like this:
ALARM ID:16554 NODE:server01 IP:10.0.0.1 FIRST:2018-05-01T05:06:32-05:00 LAST:2018-05-11T07:41:39-05:00 COUNT:2526 ...As you can see the format is different. Note the date format is 2018-05-01T05:06:32-05:00 for Horizon and 2016-11-02 08:35:29.686 for Meridian. This has cause mapping issues in a third party application the customer is using to receive and process the syslog messages forwarded by the Syslog NBI; which is why the customer would like to have the format you see on the old Meridian on newer versions.
In my opinion, the best way could be either enhance the Syslog NBI configuration to provide a format for the dates that affects the rendered messages; for example:
<syslog-northbounder-config> <enabled>true</enabled> <nagles-delay>1000</nagles-delay> <batch-size>100</batch-size> <queue-size>300000</queue-size> <date-format>yyyy-MM-dd HH:mm:ss.SSS</date-format> ...Or, provide a format while using the placeholders for dates; for example:
<message-format>ALARM ID:${alarmId} NODE:${nodeLabel} IP:${ipAddr} FIRST:${firstOccurrence::yyyy-MM-dd HH:mm:ss.SSS} LAST:${lastOccurrence::yyyy-MM-dd HH:mm:ss.SSS} COUNT:${count} UEI:${alarmUei} SEV:${severity} ${logMsg}</message-format>Note that the format is shown after "a separator" (in this case, "::").
I personally find the second option more flexible and powerful.