Uploaded image for project: 'OpenNMS'
  1. OpenNMS
  2. NMS-8030

Improve the SNMP data collection config parsing to give more flexibility to the users

    Details

      Description

      When an OpenNMS user wants to start adding customizations to the default configuration, it is very hard to avoid modifying the existing configuration files.

      A goal of every OpenNMS user should be add customizations minimizing the changes on the default files as much as possible, to make the upgrades extremely easy.

      Use Case 1

      For example, with the current code, if I want to modify a systemDef, which is a very common use case, I can exclude the original systemDef from the original file, and add a custom systemDef (with a different name) on my own data collection config group, with the content I need .. So, the configuration should look like this:

      datacollection-config.xml
      <snmp-collection name="default" snmpStorageFlag="select">
              <rrd step="300">
                  <rra>RRA:AVERAGE:0.5:1:2016</rra>
                  <rra>RRA:AVERAGE:0.5:12:1488</rra>
                  <rra>RRA:AVERAGE:0.5:288:366</rra>
                  <rra>RRA:MAX:0.5:288:366</rra>
                  <rra>RRA:MIN:0.5:288:366</rra>
              </rrd>
      
              <include-collection dataCollectionGroup="Cisco [Custom]"/>
               ...
               <include-collection dataCollectionGroup="Cisco">
                  <exclude-filter>^Cisco Routers$</exclude-filter>
              </include-collection>
              ...
      </snmp-collection>
      
      datacollection/my-cisco.xml
      <datacollection-group name="Cisco [Custom]">
            <systemDef name="Cisco Routers [Custom]">
              <sysoidMask>.1.3.6.1.4.1.9.1.</sysoidMask>
              <collect>
                <includeGroup>cisco-memory-pool</includeGroup>
                <includeGroup>cisco-router</includeGroup>
                <includeGroup>cisco-temperature</includeGroup>
                <includeGroup>cisco-voltage</includeGroup>
                <includeGroup>cisco-router-interface</includeGroup>
              </collect>
            </systemDef>
      </datacollection-group>
      

      If you forget to exclude "Cisco Routers" from cisco.xml, it will be included which means, everything defined there will be collected, even if you don't want it.

      Also, if we are talking about dozens or hundreds of systemDef, the config file will be very large.

      Unfortunately, the WebUI doesn't support the exclusion filters which means, all the changes have to be done manually.

      Proposal

      A more "logic" idea could be, add a copy of the Cisco Routers systemDef on my custom datacollection config group, and just by including a reference to the group before "Cisco" (i.e. cisco.xml), should be enough to tell the parser to use that group and not the one from cisco.xml. Unfortunately, this won't work with the current code.

      Use Case 2

      Now, imagine that you want to do a similar thing but for a MibObj group. There are two ways with the current code:

      1) Add a copy of the group directly to the SNMP Collection definition, to be sure that nothing from the data collection group files will be added.

      This is because whatever exist on the SNMP Collection has more privileges than the dynamically added groups from the data collection group files.

      This option works, if you have one SNMP Collection, otherwise, you have to copy/paste your custom changes across all your SNMP Collection which is definitively not maintainable.

      2) Go ahead and modify the appropriate data collection config group file, and deal with OpenNMS upgrades later. This is basically the action that all uses do.

      The problem with this is when you want to upgrade OpenNMS. If you're on a RPM based OS, you'll see a .rpmnew for the modified file, because of course it was modified. That means, you have to perform the merges between the files manually which is a pain. A similar thing will happen with the Debian packages.

      Of course, a user can just ignore the new content and use whatever is working at the moment. This is completely valid, I don't have objections.

      That being said, I think we can have a nicer alternative.

      Proposal

      When adding the groups defined on a particular systemDef, instead of looking for it on all the available groups, look at the group that is being processed first. If it has the the MibObj group, use that. If not, go ahead and find it on all the other groups.

      That way, it is very easy to override an existing MibObj group by just adding it to your custom data collection group file, and using it on your custom systemDef.

      Now, what if I want to override a MibObj group without creating systemDefs ? Well, this use case can be covered by creating a dummy systemDef like the following, with a reference to the modified MibObj group:

       <systemDef name="Mock SystemDef, to force the load my custom groups">
           <sysoid>.0.0.0.0</sysoid>
           <collect>
               <includeGroup>cisco-memory-pool</includeGroup>
           </collect>
       </systemDef>
      

      As you can see, the name is very explicit, and the sysoid should be a well formed but invalid OID. By invalid I mean, a sequence that for sure doesn't exist on the real world, because the idea is to tell the parser to add cisco-memory-pool from my custom definition before cisco.xml is processed. That way, I can be sure that my version will be used.

      Solution

      If we implement this 2 proposals, we're covering a big percentage of all the possible changes that a user could introduce on the SNMP data collection config, without the need of modifying the existing files, or even deleting the existing files.

        Attachments

          Activity

            People

            • Assignee:
              agalue Alejandro Galue
              Reporter:
              agalue Alejandro Galue
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: