Trapd is not able to process SNMPv3 INFORMs

I have made many tests in sending snmp v3 traps and informs to opennms trapd service and they were all discarded.
Just looking at the java code it seems to me that is impossible to detect the snmp v3 traps.
Isn't it?

I changed the title of the issue because the original description is a consequence of the fact that the SnmpStrategy for SNMP4J is not able to properly process SNMPv3 Traps on the current code (tested on 1.8 and 1.10 branches), because the session doesn't contains the USM users to be used to decode and authenticate traps.

I'm currently working on a solution for this issue.

Alejandro,
let me state some notes about this implementation:

1) where to put authentication information,
the trapd configuration file allow you to set only the listening port.
Well the only place seem to be snmp-config.xml.

2) Authentication of incoming snmp v3 INFORMS is ok because the EngineID (that is required for authentication) is the engineId of sending Agent

3) Authentication of incoming snmp v3 TRAPS require that the agent is able to know the "Snmp4J + OpenNMS" engineID.

See here for more details:

http://www.net-snmp.org/tutorial/tutorial-5/commands/snmptrap-v3.html

I have deployed a simple method that simply return the "Snmp4J-OpenNMS" engineID in the class:

SnmpTrapHelper.java: * EngineID that needs to be known
SnmpTrapHelper.java: public String getLocalEngineID() {
SnmpTrapHelper.java: return "0x"+SnmpUtils.getLocalEngineID();

More I also have deployed support for Snmpv3 traps and informs under the SnmpStrategy interface and in particular for Snmp4J.
JoeSnmp try an Unsupported Exception.

Hope this helps

Hope this helps.

Hey Antonio, please correct me if I'm wrong.

The SNMPv3 information from snmp-config.xml is the configuration needed to retrieve data from external devices.

The SNMPv3 configuration that should be part of trapd-configuration.xml is the configuration needed to receive data (authenticate and decode) from external devices.

I made some testing with something like this:

<trapd-configuration snmp-trap-port="162" new-suspect-on-trap="true">
<snmpv3-user security-name="alejandro" auth-passphrase="galue" auth-protocol="MD5" privacy-passphrase="galue" privacy-protocol="DES"/>
<snmpv3-user security-name="testUser" auth-passphrase="0p3nNMSv3" auth-protocol="MD5" privacy-passphrase="0p3nNMSv3" privacy-protocol="DES"/>
</trapd-configuration>

That allows me to send SNMP3 traps from external devices like this:

snmptrap -v 3 -n "" -a MD5 -A galue -x DES -X galue -l authPriv -u alejandro 192.168.0.5 0 coldStart.0

192.168.0.5 is the IP of the OpenNMS server.
The snmptrap was executed on a Linux box monitored by OpenNMS.

The required changes on Trapd are associated with the SNMP4J session created to handle traps inside the method registerForTraps from Snmp4JStrategy.java. If I add something like this to that method:

for (SnmpV3User user : snmpUsers) {
SnmpAgentConfig config = new SnmpAgentConfig();
config.setVersion(SnmpConfiguration.VERSION3);
config.setSecurityName(user.getSecurityName());
config.setAuthProtocol(user.getAuthProtocol());
config.setAuthPassPhrase(user.getAuthPassPhrase());
config.setPrivProtocol(user.getPrivProtocol());
config.setPrivPassPhrase(user.getPrivPassPhrase());
Snmp4JAgentConfig agentConfig = new Snmp4JAgentConfig(config);
snmp.getUSM().addUser(
agentConfig.getSecurityName(),
new UsmUser(
agentConfig.getSecurityName(),
agentConfig.getAuthProtocol(),
agentConfig.getAuthPassPhrase(),
agentConfig.getPrivProtocol(),
agentConfig.getPrivPassPhrase()
)
);
}

where snmpUsers is the list of users defined on trapd-configuration.xml. Every SnmpV4User is a simple POJO with strings. I used Snmp4JAgentConfig to properly add UsmUsers to the SNMP4J Session Object.

Now I'm able to receive SNMPv3 in OpenNMS.

The required code changes works for 1.8 and 1.10 (I tested both)

Thoughts?

Sorry to interject while you're busy implementing a solution for this, but I have a query about the bug description. The description suggests that there is only an issue if the device in question is not already managed ("SNMPv3 traps received from IP addresses not managed by openNMS..."). However, I'm having the issue even when the IP address is managed by OpenNMS. (i.e. I have a device that I can successfully GetRequest using v3 auth SHA priv AES, yet traps from the same device are ignored with "1.3.6.1.6.3.15.1.1.3.0".)

So that I don't waste others' time on the mailing list - and for the benefit of others that find this issue - please could you tell me, are v3 traps broken until this issue is fixed or should I expect them to work?

Thanks.

Jira made a mess with the portion of the code from my last comment.

I included a diff patch with my proposed changes.

The patch has been created for 1.10, but it should work with 1.8

I tested this solution with the snmptrap command and with the SnmpV3TrapBuilder.

I'll appreciate if anyone could test it.

Bob, that is the reason of why I changed the subject of this issue, because it doesn't work on any ways (monitored or non-monitored nodes)

Thanks for clarifying. I'll get out of the way and let you deal with the issue. =)

Alejandro it is surprising that you are able to send trap to opennms without the engineId for authentication.
You should also add the ability to set the engioneId for the USM.

Alejandro it is surprising that you are able to send trap to opennms without the engineId for authentication.
You added the ability to set the engineId for the USM but it is never used.
BTW you should commit the changes in 1.10.

I'm interested in understanding if snmp4j does not use the engineid and securityname for authentication.
Both are used to authenticate the snmp v3 packet (meaning the trap).

Antonio, I'm agree with you.

I updated the code to handle an engineId per user if that exists on trapd-configuration.xml, like this:

<trapd-configuration snmp-trap-port="162" new-suspect-on-trap="true">
<snmpv3-user security-name="agalue" auth-passphrase="0p3nNMSv3" auth-protocol="MD5" privacy-passphrase="0p3nNMSv3" privacy-protocol="DES" engine-id="0102030405"/>
<snmpv3-user security-name="user2" auth-passphrase="0p3nNMSv3" auth-protocol="MD5" privacy-passphrase="0p3nNMSv3" privacy-protocol="DES"/>
</trapd-configuration>

The relevant part of the code change from Snmp4JStrategy is this:

if (user.getEngineId() == null) {
snmp.getUSM().addUser(agentConfig.getSecurityName(), usmUser);
} else {
snmp.getUSM().addUser(agentConfig.getSecurityName(), new OctetString(user.getEngineId()), usmUser);
}

Now it doesn't work, so I left the code as the first version.

With the first version (check the attached diff patch), the following commands work as expected:

snmptrap -v 3 -n "" -a MD5 -A 0p3nNMSv3 -x DES -X 0p3nNMSv3 -l authPriv -u agalue 192.168.0.5 0 coldStart.0
snmptrap -v 3 -n "" -e 0x0102030405 -a MD5 -A 0p3nNMSv3 -x DES -X 0p3nNMSv3 -l authPriv -u agalue 192.168.0.5 0 coldStart.0

I'm totally agree with you that only the second one should be valid, so I suspect that there is something wrong on SNMP4J.

I did that with snmptrapd (from Net-SNMP), and the only one that works is the second one (i.e., the engineId is required).

So I will commit the changes for 1.10, but I'll let the issue open for future review.

The changes have been committed in revision 062a2de1cd354558df53bd8d8bab4c5ec999a5eb for 1.10

Alajandro I agree with you. It is a trouble with snmp4j they do not use the engineId for authentication.

The changes have been cherry-picked and updated in revision 7d7fd2c316b2ad364772a058f19526804caf20b0 for 1.8

I am running version 1.10 and still have the issue described in this bug. I receive traps for a few minutes and then they stop. Am I missing something?

Based on what I recall from my tests, this is related with a problem of the remote SNMP agent.

Here is an interesting link:
http://comments.gmane.org/gmane.network.snmp4j.general/521

There are more details about the discussion about this problem in the archives of out mailing list (opennms-discuss).

Thank you for your response. I found this thread in the opennms-discuss list but don't see any resolution.
http://sourceforge.net/mailarchive/forum.php?thread_name=BANLkTikR%3Da5gUoRNg9_AG6Syo6TgzMgK6A%40mail.gmail.com&forum_name=opennms-discuss

I have 1.10.1 installed and seem to have the same issue. if I issue the command from the opennms machine:

1. snmptrap -v 3 -n "" -a MD5 -A galue -x DES -X galue -l authPriv -u alejandro 192.168.0.5 0 coldStart.0

I see the event but if I do it from another machine on the net I don't see the event. Maybe something wrong with my snmp-conf file? other machine on 10.0.2.x net and in that range.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<snmp-config retry="1" timeout="1800" read-community="public" version="v2c" xmlns="http://xmlns.opennms.org/xsd/config/snmp">
<definition read-community="public" version="v2c">
<range begin="10.0.2.1" end="10.0.2.254"/>
</definition>
<definition version="v3"
security-name="opennmsUser"
auth-passphrase="passowrd"
auth-protocol="MD5"
privacy-passphrase="password"
privacy-protocol="DES">
<range begin="10.0.2.1" end="10.0.2.254"/>
</definition>

</snmp-config>

Sorry if I revisit an old topic, but the sending of SNMPv3 informs still doesn't work for me on OpenNMS.

First of all, here's the relevant part of RFC2274, section 1.5.1:

When an SNMP message contains a payload which expects a response (for example a Get, GetNext, GetBulk, Set or Inform PDU), then the receiver of such messages is authoritative. When an SNMP message contains a payload which does not expect a response (for example an SNMPv2-Trap, Response or Report PDU), then the sender of such a message is authoritative.

So the process for sending Traps and Informs is fundamentally different in that for a Trap the engine ID of the agent is used, while for an Inform the engine ID of the manager is required. Since the latter is initially not known to the agent, it uses a simple workaround by sending a GetRequest to the manager (on port 162!), which is answered with a Report PDU containing the manager's engine ID:

16:53:25.321232 IP ardbeg.hindenburgring.com.57046 > orcus3.hindenburgring.com.snmptrap:  F=r U= E=  C= GetRequest(14) 
16:53:25.329159 IP orcus3.hindenburgring.com.snmptrap > ardbeg.hindenburgring.com.57046:  F= U= E= 0x800x000x1F0x880x800xD40xB20x270x0D0x800xC50x9F0x500x000x000x000x00 C= Report(31)  S:snmpUsmMIB.usmMIBObjects.usmStats.usmStatsUnknownEngineIDs.0=1

Now that the manager's (which is Net-SNMP's snmptrapd in this case) engine ID is known, the actual inform can be sent, and the resulting Response PDU is returned and can be properly decoded by the agent:

16:53:25.329538 IP ardbeg.hindenburgring.com.57046 > orcus3.hindenburgring.com.snmptrap:  F=apr U=opennms-notify [!scoped PDU]12_a3_b6_02_47_f2_b1_4a_cd_aa_e2_2a_9f_49_c4_39_6a_87_f3_dc_e0_96_7e_76_dd_17_0d_4c_2e_2d_bd_10_6d_03_8d_ac_86_99_be_09_99_06_41_d2_45_aa_cd_35_05_66_e8_bf_66_9e_c9_8c_24_81_28_e8_77_37_37_5f_db_c1_48_55_5f_5f_8e_29_81_bb_c7_f4_3b_df_3c_09_22_1c_4f
16:53:25.343953 IP orcus3.hindenburgring.com.snmptrap > ardbeg.hindenburgring.com.57046:  F=ap U=opennms-notify [!scoped PDU]c8_79_c8_25_c9_28_12_13_74_28_c2_f9_e1_99_57_22_9a_97_71_66_53_b7_82_51_65_a4_77_32_0d_ba_f2_47_ba_97_09_df_01_6b_37_49_76_a2_f5_d0_fc_e7_eb_fd_18_6d_28_c8_eb_a2_da_02_85_4d_01_76_ce_0c_d8_2b_3d_1a_de_d3_12_4e_8c_8f_6d_a3_fa_8b_4d_be_24_2c_f6_a1_b1

At this point, everything is OK.

Unfortunately, the exact same approach does not work if the manager is OpenNMS:

17:17:00.601875 IP ardbeg.hindenburgring.com.63175 > orcus3.hindenburgring.com.snmptrap:  F=r U= E=  C= GetRequest(14) 
17:17:00.603517 IP orcus3.hindenburgring.com.snmptrap > ardbeg.hindenburgring.com.63175:  F= U= E=  C= Report(28)   S:snmpUsmMIB.usmMIBObjects.usmStats.usmStatsUnknownEngineIDs.0=0

Note that the engine ID field is missing from the Report PDU, and that the value of the usmStatsUnknownEngineIDs.0 OID is 0, not 1 (not sure if the latter matters).

Since the agent does not know the engine ID of the manager, it can't decode the Response PDU and keeps retrying to send the Inform packet. OpenNMS, strangely enough, gets all the Informs and creates an event for each of them:

17:17:00.603812 IP ardbeg.hindenburgring.com.63175 > orcus3.hindenburgring.com.snmptrap:  F=apr U=opennms-notify [!scoped PDU]a3_a2_09_6c_dc_4b_d9_7f_5d_3c_30_42_04_8e_61_5a_c4_81_4f_d2_58_09_32_d1_dc_a4_40_57_2a_00_c2_c5_58_5d_ea_f6_17_24_51_66_a1_09_1b_27_15_90_75_01_b2_30_7a_99_70_a3_ba_e6_16_9d_f9_b4_1e_24_79_39_9f_cf_70_c9_a8_4e_4b_8f_43_4c_13
17:17:00.668067 IP orcus3.hindenburgring.com.snmptrap > ardbeg.hindenburgring.com.63175:  F=ap U=opennms-notify [!scoped PDU]31_09_10_1d_40_77_a1_67_63_b9_7d_45_93_1d_a6_10_c3_8f_44_1e_fe_25_14_2d_90_e4_3f_59_18_e6_76_b1_4c_b9_25_49_99_9b_f3_96_a0_76_c0_01_76_3d_f5_73_65_1e_50_3e_c8_78_d1_65_b8_a4_5e_6a
17:17:01.605098 IP ardbeg.hindenburgring.com.63175 > orcus3.hindenburgring.com.snmptrap:  F=apr U=opennms-notify [!scoped PDU]05_2a_d7_ff_36_48_4c_96_3a_2c_11_cf_19_d1_0e_96_9c_48_7e_31_d6_53_88_4d_a6_a6_e2_70_64_47_e8_be_0c_c3_70_06_37_f4_5f_b5_ba_05_da_93_60_d9_a4_c8_4d_fa_01_56_5a_9e_16_ff_dc_10_d3_31_11_2a_1e_84_e6_08_93_42_d3_c0_b5_43_98_d5_b2
17:17:01.606544 IP orcus3.hindenburgring.com.snmptrap > ardbeg.hindenburgring.com.63175:  F=ap U=opennms-notify [!scoped PDU]68_92_d4_eb_5a_5a_7e_6d_09_e1_11_de_73_5c_e9_82_66_85_7b_26_84_7d_3d_71_7f_06_9f_f9_8f_d5_cd_f4_d1_32_bd_8f_b2_d9_58_19_f7_b1_d6_c0_65_b2_e0_81_27_3b_33_9c_d0_90_35_e7_a9_8c_1b_71
17:17:02.606220 IP ardbeg.hindenburgring.com.63175 > orcus3.hindenburgring.com.snmptrap:  F=apr U=opennms-notify [!scoped PDU]99_0d_65_2a_0b_e0_aa_fd_b6_f1_29_bc_a6_1a_20_a7_81_82_10_62_fc_7e_4f_06_eb_0e_c9_7f_96_ec_c4_b4_bd_8d_c1_b0_a4_e2_a4_50_0b_8c_85_f3_d5_7d_39_3b_46_3d_65_6e_48_9a_fb_83_de_19_da_51_07_95_4e_0b_e2_b1_5b_72_33_e9_04_2a_42_c1_11
17:17:02.608147 IP orcus3.hindenburgring.com.snmptrap > ardbeg.hindenburgring.com.63175:  F=ap U=opennms-notify [!scoped PDU]48_55_cd_9b_23_10_98_34_0c_b2_77_65_7b_53_65_b3_78_aa_82_9e_b5_8b_c1_0f_f1_46_68_b9_25_95_4e_3e_fc_cb_6f_42_2d_91_70_aa_18_5e_d7_d1_75_ef_80_76_f6_e2_4c_f1_e0_85_a6_f4_1d_5c_af_f0
17:17:03.606688 IP ardbeg.hindenburgring.com.63175 > orcus3.hindenburgring.com.snmptrap:  F=apr U=opennms-notify [!scoped PDU]4e_67_31_55_57_f8_78_fc_a9_86_d4_5a_fa_8d_51_6e_6b_0b_fe_39_02_00_40_bb_1f_09_e2_d1_69_7f_35_5c_14_8d_ee_55_4d_10_e3_6a_17_17_7c_99_f8_65_c4_82_75_c9_3a_1e_15_73_46_30_a3_e4_bc_7f_ae_f8_96_8b_7c_87_06_d4_e9_06_53_9f_d9_f3_68
17:17:03.609068 IP orcus3.hindenburgring.com.snmptrap > ardbeg.hindenburgring.com.63175:  F=ap U=opennms-notify [!scoped PDU]ca_df_27_81_23_2f_0c_ea_e4_42_c0_6e_dc_5b_d8_f0_9f_d3_bf_b9_35_91_8d_4e_33_69_77_79_5c_1d_76_0e_f9_22_c1_b4_7a_66_ae_97_2d_2c_c4_97_3d_2e_52_17_76_8a_77_26_c7_91_b2_f6_51_56_ff_81
17:17:04.607970 IP ardbeg.hindenburgring.com.63175 > orcus3.hindenburgring.com.snmptrap:  F=apr U=opennms-notify [!scoped PDU]0b_d9_df_93_31_69_61_79_0b_0b_7b_47_62_aa_3e_bd_82_3d_ec_6d_69_eb_e9_ba_dc_9c_27_6e_76_7b_10_b4_dc_36_f3_a1_5a_65_54_95_46_bc_1e_56_fc_16_f2_f9_86_e7_e9_c8_27_0d_40_e3_c4_50_05_02_11_c5_cb_05_45_64_02_7a_53_cb_c2_5f_c3_9a_ea
17:17:04.673782 IP orcus3.hindenburgring.com.snmptrap > ardbeg.hindenburgring.com.63175:  F=ap U=opennms-notify [!scoped PDU]37_06_b3_df_3d_45_a5_33_f9_00_47_ed_b1_98_6e_90_e4_e5_5a_19_c0_f8_33_e3_b0_99_66_70_64_55_b1_9f_6b_a3_4d_2a_47_37_24_e2_52_0b_29_97_c0_25_0b_a2_30_04_e3_c3_56_5d_f3_79_d3_b9_b4_bb
17:17:05.609245 IP ardbeg.hindenburgring.com.63175 > orcus3.hindenburgring.com.snmptrap:  F=apr U=opennms-notify [!scoped PDU]ee_35_f5_5b_12_3f_cd_2b_1a_c8_f3_18_b8_a4_b7_c1_d5_1d_d1_19_2e_6f_92_4f_58_d4_1f_b7_e5_4e_74_0c_ae_1a_56_1e_53_4c_f2_ef_c4_1f_6d_2d_70_d2_a6_92_bf_d1_bd_92_b1_ab_7e_65_6d_ba_0d_00_fd_96_af_ce_65_b0_69_1d_e4_0f_eb_d5_38_86_86
17:17:05.610885 IP orcus3.hindenburgring.com.snmptrap > ardbeg.hindenburgring.com.63175:  F=ap U=opennms-notify [!scoped PDU]a8_cb_d7_db_78_13_fa_b7_62_5f_75_83_e2_b8_36_e1_72_ac_3f_24_74_79_df_71_0f_81_bc_7a_ff_35_ba_45_c2_c9_86_a1_5a_e1_e7_08_68_3e_9e_71_96_8b_04_34_ad_3c_e6_62_66_78_84_cb_61_bf_ef_ba

To summarise the problem:

• SNMPv3 traps can be sent without any problem, as the local engine ID is used for encrypting the scoped PDU.
• SNMP4J does not return its engine ID in the Response PDU for the GetRequest issued by the agent (which is Net-SNMP's snmpinform CLI tool, by the way).
• The Informs are decoded and interpreted anyway, just the Response PDUs are not decodable by snmpinform, which leads to retransmits.

If there were any way to get SNMP4J to return the engine ID, I guess the problem would be solved immediately. Am I missing something, or is this a bug in SNMP4J?

<?xml version="1.0"?>
<trapd-configuration snmp-trap-port="162" new-suspect-on-trap="false">

    <snmpv3-user security-name="opennms-notify"
                 auth-passphrase="nkKgYe6oBF6u7RrtvqV6bR"
                 auth-protocol="SHA"
                 privacy-passphrase="XJe74Z9LCeiywe3ewTvrqg"
                 privacy-protocol="AES" />

</trapd-configuration>

snmpinform -v3 \
           -u opennms-notify \
           -l authPriv \
           -a SHA \ 
           -A nkKgYe6oBF6u7RrtvqV6bR \
           -x AES \
           -X XJe74Z9LCeiywe3ewTvrqg \
           orcus3.hindenburgring.com \
           "" \
           .1.3.6.1.4.1.15904.1.1.2

Regards,

Peter.

Sorry, I just added a comment recently but what I was actually intending was to re-open the issue. At least for SNMPv3 informs, it does not seem to be resolved.

I have some more information on this, and steps to reproduce.

Actually the problem can be isolated with SNMPv3 informs, traps work fine. First of all, here is the method to reproduce the problem.

1. Set up an SNMPv3 user in etc/trapd-configuration.xml (and restart OpenNMS afterwards)
[code]
<snmpv3-user security-name="OpenNMS-Inform"
security-level="3"
auth-protocol="SHA"
auth-passphrase="opennms-inform-auth"
privacy-protocol="AES"
privacy-passphrase="opennms-inform-priv" />
[code]

2. Determine the IP address of the OpenNMS box in hexadecimal form. By default, OpenNMS uses the IP address to generate the security engine ID, so if, for example, the actual IP address is 192.168.42.74, the hexadecimal form would be C0A82A4A. The engine ID is that IP address, prefixed by the enterprise identifier 800016b5 and the engine ID format identifier 01, which makes it 800016b501C0A82A4A in this example.

3. Send an arbitrary SNMPv3 inform using the Net-SNMP snmpinform tool:
[code]
snmpinform -v3 -u OpenNMS-Inform \
-e 800016b501C0A82A4A \
-l authPriv \
-a SHA -A opennms-inform-auth \
-x AES -X opennms-inform-priv \
192.168.42.74 0 1.3.6.1.4.1.15904.1.1.2.0.1
[code]
The OID of the inform doesn't matter. Just the authentication parameters and the engine ID must match.

Two things will happen:

• The SNMP inform will arrive in OpenNMS 5 times (more or less, depending on the setting for 'retries' in the Net-SNMP configuration - 5 is default)

• The CLI tool will receive an error message and a timeout:
  [code]
  No log handling enabled - using stderr logging
  snmpinform: Timeout (plaintext scopedPDU header type A2: s/b 30)
  [code]

I dug into this a bit deeper using WireShark, and the problem seems to be that the packet from the Net-SNMP client tool is authenticated and encrypted properly, while the Response PDU from OpenNMS/SNMP4J is correctly authenticated, but the encrypted PDU is corrupt.

I'll attach a screen shot and the capture file shortly.

Another update to this.

I tried to leave the encryption part out of the picture by modifying the OpenNMS setup so that the user does not require encryption. Unfotunately, the SNMP inform request fails again, and in this case it is easy to see why: The response PDU is missing from the SNMPv3 packet ...

I'll add another screen shot and another packet dump.