OpenNMS

Provisiond ignores foreign source category policies on subsequent synchronizations

Details

Description

(Bug updated from the original submission - I can now reproduce this at will with a simpler test case without involving DNS import)

Here is my concept:

- Define a provisioning group/foreign source with associated policies that place imported nodes into categories
- Use the requisition to provision additional categories on some nodes as appropriate

Here's what's happening:

- I build the provisioning group, and define policies for category membership
- I import the nodes
- Everything looks OK, categories appear fine once the very first nodeScanCompleted event hits
- (BUG) At some later point, generally after adding some more nodes, all my policy-defined categories disappear
- (BUG) Editing the policy and/or manually synchronizing the group does not bring the categories back

Attached is a detailed set of evidence proving that the problem exists in my environment even after all provisioning has completed. (In an earlier discussion with Matt in the comments on my initial submission of this bug, I thought this might have been related to me simply not waiting for the nodeScanCompleted event, but the evidence disproves this.)

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Matt Brozowski added a comment -

I looked recently at a bug similar to this and found that applying policies worked fine to add nodes to categories. However, it should be noted that the policies are applied during the 'scan' phase rather than during the 'import' phase so group membership may be delayed since it has to finish scanning.

Can you verify that you are not just looking 'too soon' and that the categories are added later?

Show
Matt Brozowski added a comment - I looked recently at a bug similar to this and found that applying policies worked fine to add nodes to categories. However, it should be noted that the policies are applied during the 'scan' phase rather than during the 'import' phase so group membership may be delayed since it has to finish scanning. Can you verify that you are not just looking 'too soon' and that the categories are added later?
Hide
Permalink
Andy Ellsworth added a comment -

I was actually just scratching my head trying to figure out why my categories are now showing up, and attempting to reproduce my bug again. The delay for the scans to complete does seem to fit what I'm seeing - I will watch the provisiond logs to confirm. Thanks for the info!

Show
Andy Ellsworth added a comment - I was actually just scratching my head trying to figure out why my categories are now showing up, and attempting to reproduce my bug again. The delay for the scans to complete does seem to fit what I'm seeing - I will watch the provisiond logs to confirm. Thanks for the info!
Hide
Permalink
Andy Ellsworth added a comment -

I'm going to close out this bug, as I can't reproduce it now, and Matt's explanation is the most likely reason why.

Show
Andy Ellsworth added a comment - I'm going to close out this bug, as I can't reproduce it now, and Matt's explanation is the most likely reason why.
Hide
Permalink
Andy Ellsworth added a comment -

Bug went away (i.e. policies eventually kicked in). I think the key is, as Matt said, to wait until the post-import scanning is complete.

Show
Andy Ellsworth added a comment - Bug went away (i.e. policies eventually kicked in). I think the key is, as Matt said, to wait until the post-import scanning is complete.
Hide
Permalink
Andy Ellsworth added a comment -

Bug came back, and I can reproduce it now - overhauled the bug description and title to reflect the new test case.

Show
Andy Ellsworth added a comment - Bug came back, and I can reproduce it now - overhauled the bug description and title to reflect the new test case.
Hide
Permalink
Andy Ellsworth added a comment -

This bug seems somewhat related to NMS-5059.

Show
Andy Ellsworth added a comment - This bug seems somewhat related to NMS-5059 .
Hide
Permalink
Andy Ellsworth added a comment -

Not sure if this is indicative of the problem, or a red herring, but found this in the provisiond logs, seems relevant:

2011-11-18 22:41:08,776 INFO [DefaultUDPTransportMapping_**MASKED**/0] DefaultProvisionService: Configured plugin is not appropropriate for policy class interface org.opennms.netmgt.provision.IpInterfacePolicy: org.opennms.netmgt.provision.persist.foreignsource.PluginConfig@2414c3da[name=Category1,plugin-class=org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy,parameters=category=D-Windows/matchBehavior=ALL_PARAMETERS]
2011-11-18 22:41:08,826 DEBUG [DefaultUDPTransportMapping_**MASKED**/0] DefaultPluginRegistry: Found bean null with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.IpInterfacePolicy
2011-11-18 22:41:08,826 DEBUG [scanExecutor-12] DefaultPluginRegistry: Found bean org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy@2dc5a1a8 with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.NodePolicy
2011-11-18 22:41:08,826 INFO [DefaultUDPTransportMapping_**MASKED**/0] DefaultProvisionService: Configured plugin is not appropropriate for policy class interface org.opennms.netmgt.provision.IpInterfacePolicy: org.opennms.netmgt.provision.persist.foreignsource.PluginConfig@6426fe03[name=Category3,plugin-class=org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy,parameters=category=P-ICMP-1min/matchBehavior=ALL_PARAMETERS]
2011-11-18 22:41:08,826 DEBUG [scanExecutor-12] DefaultPluginRegistry: Found bean org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy@31f04de2 with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.NodePolicy
2011-11-18 22:41:08,826 DEBUG [DefaultUDPTransportMapping_**MASKED**/0] DefaultPluginRegistry: Found bean null with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.IpInterfacePolicy
2011-11-18 22:41:08,826 INFO [DefaultUDPTransportMapping_**MASKED**/0] DefaultProvisionService: Configured plugin is not appropropriate for policy class interface org.opennms.netmgt.provision.IpInterfacePolicy: org.opennms.netmgt.provision.persist.foreignsource.PluginConfig@7f95f87d[name=Category4,plugin-class=org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy,parameters=category=D-MIB2InterfaceStats/matchBehavior=ALL_PARAMETERS]
2011-11-18 22:41:08,827 DEBUG [scanExecutor-12] DefaultPluginRegistry: Found bean org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy@13b2cabe with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.NodePolicy
2011-11-18 22:41:08,827 DEBUG [DefaultUDPTransportMapping_**MASKED**/0] DefaultPluginRegistry: Found bean null with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.IpInterfacePolicy
2011-11-18 22:41:08,827 INFO [DefaultUDPTransportMapping_**MASKED**/0] DefaultProvisionService: Configured plugin is not appropropriate for policy class interface org.opennms.netmgt.provision.IpInterfacePolicy: org.opennms.netmgt.provision.persist.foreignsource.PluginConfig@31bc1af4[name=Category5,plugin-class=org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy,parameters=category=D-MIB2InterfaceStats-1min/matchBehavior=ALL_PARAMETERS]
2011-11-18 22:41:08,827 DEBUG [scanExecutor-12] DefaultPluginRegistry: Found bean org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy@614cd632 with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.NodePolicy
2011-11-18 22:41:08,827 DEBUG [DefaultUDPTransportMapping_**MASKED**/0] DefaultPluginRegistry: Found bean null with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.IpInterfacePolicy
2011-11-18 22:41:08,827 DEBUG [scanExecutor-12] DefaultPluginRegistry: Found bean org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy@76dd3979 with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.NodePolicy
2011-11-18 22:41:08,827 INFO [DefaultUDPTransportMapping_**MASKED**/0] DefaultProvisionService: Configured plugin is not appropropriate for policy class interface org.opennms.netmgt.provision.IpInterfacePolicy: org.opennms.netmgt.provision.persist.foreignsource.PluginConfig@56e96caa[name=Category6,plugin-class=org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy,parameters=category=M-Servers/matchBehavior=ALL_PARAMETERS]

Show
Andy Ellsworth added a comment - Not sure if this is indicative of the problem, or a red herring, but found this in the provisiond logs, seems relevant: 2011-11-18 22:41:08,776 INFO [DefaultUDPTransportMapping_**MASKED**/0] DefaultProvisionService: Configured plugin is not appropropriate for policy class interface org.opennms.netmgt.provision.IpInterfacePolicy: org.opennms.netmgt.provision.persist.foreignsource.PluginConfig@2414c3da [name=Category1,plugin-class=org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy,parameters=category=D-Windows/matchBehavior=ALL_PARAMETERS] 2011-11-18 22:41:08,826 DEBUG [DefaultUDPTransportMapping_**MASKED**/0] DefaultPluginRegistry: Found bean null with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.IpInterfacePolicy 2011-11-18 22:41:08,826 DEBUG [scanExecutor-12] DefaultPluginRegistry: Found bean org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy@2dc5a1a8 with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.NodePolicy 2011-11-18 22:41:08,826 INFO [DefaultUDPTransportMapping_**MASKED**/0] DefaultProvisionService: Configured plugin is not appropropriate for policy class interface org.opennms.netmgt.provision.IpInterfacePolicy: org.opennms.netmgt.provision.persist.foreignsource.PluginConfig@6426fe03 [name=Category3,plugin-class=org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy,parameters=category=P-ICMP-1min/matchBehavior=ALL_PARAMETERS] 2011-11-18 22:41:08,826 DEBUG [scanExecutor-12] DefaultPluginRegistry: Found bean org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy@31f04de2 with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.NodePolicy 2011-11-18 22:41:08,826 DEBUG [DefaultUDPTransportMapping_**MASKED**/0] DefaultPluginRegistry: Found bean null with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.IpInterfacePolicy 2011-11-18 22:41:08,826 INFO [DefaultUDPTransportMapping_**MASKED**/0] DefaultProvisionService: Configured plugin is not appropropriate for policy class interface org.opennms.netmgt.provision.IpInterfacePolicy: org.opennms.netmgt.provision.persist.foreignsource.PluginConfig@7f95f87d [name=Category4,plugin-class=org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy,parameters=category=D-MIB2InterfaceStats/matchBehavior=ALL_PARAMETERS] 2011-11-18 22:41:08,827 DEBUG [scanExecutor-12] DefaultPluginRegistry: Found bean org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy@13b2cabe with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.NodePolicy 2011-11-18 22:41:08,827 DEBUG [DefaultUDPTransportMapping_**MASKED**/0] DefaultPluginRegistry: Found bean null with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.IpInterfacePolicy 2011-11-18 22:41:08,827 INFO [DefaultUDPTransportMapping_**MASKED**/0] DefaultProvisionService: Configured plugin is not appropropriate for policy class interface org.opennms.netmgt.provision.IpInterfacePolicy: org.opennms.netmgt.provision.persist.foreignsource.PluginConfig@31bc1af4 [name=Category5,plugin-class=org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy,parameters=category=D-MIB2InterfaceStats-1min/matchBehavior=ALL_PARAMETERS] 2011-11-18 22:41:08,827 DEBUG [scanExecutor-12] DefaultPluginRegistry: Found bean org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy@614cd632 with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.NodePolicy 2011-11-18 22:41:08,827 DEBUG [DefaultUDPTransportMapping_**MASKED**/0] DefaultPluginRegistry: Found bean null with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.IpInterfacePolicy 2011-11-18 22:41:08,827 DEBUG [scanExecutor-12] DefaultPluginRegistry: Found bean org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy@76dd3979 with name org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy of type interface org.opennms.netmgt.provision.NodePolicy 2011-11-18 22:41:08,827 INFO [DefaultUDPTransportMapping_**MASKED**/0] DefaultProvisionService: Configured plugin is not appropropriate for policy class interface org.opennms.netmgt.provision.IpInterfacePolicy: org.opennms.netmgt.provision.persist.foreignsource.PluginConfig@56e96caa [name=Category6,plugin-class=org.opennms.netmgt.provision.persist.policies.NodeCategorySettingPolicy,parameters=category=M-Servers/matchBehavior=ALL_PARAMETERS]
Hide
Permalink
Andy Ellsworth added a comment -

I think I may have figured this one out - one of the surveillance categories that I was trying to assign via foreign source policy (M-Servers) did not exist, and appears to have been causing this behavior. Creating the missing category through the Web UI followed by some combination of OpenNMS restarts and provisioning group synchronizations seems to have cleared this up.

I will let this ride for a few days and make sure it doesn't recur, and then see if I can generate a test case for more graceful handling of trying to assign undefined categories via provisiond policy.

Show
Andy Ellsworth added a comment - I think I may have figured this one out - one of the surveillance categories that I was trying to assign via foreign source policy (M-Servers) did not exist, and appears to have been causing this behavior. Creating the missing category through the Web UI followed by some combination of OpenNMS restarts and provisioning group synchronizations seems to have cleared this up. I will let this ride for a few days and make sure it doesn't recur, and then see if I can generate a test case for more graceful handling of trying to assign undefined categories via provisiond policy.
Hide
Permalink
Benjamin Reed added a comment -

Please do; we should be either a) creating categories, or b) giving a reasonable error message.

Not sure which is more correct for the Provisiond use-case, but quietly failing is definitely not.

Show
Benjamin Reed added a comment - Please do; we should be either a) creating categories, or b) giving a reasonable error message. Not sure which is more correct for the Provisiond use-case, but quietly failing is definitely not.

People

  • Assignee:
    Matt Brozowski
    Reporter:
    Andy Ellsworth
Vote (1)
Watch (1)

Dates

  • Created:
    Updated: