Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Blocker
    • Resolution: Won't Fix
    • Affects Version/s: 1.12.9
    • Fix Version/s: 14.0.0
    • Component/s: Data Collection - HTTP
    • Security Level: Default (Default Security Scheme)
    • Labels:
      None

      Description

      The HttpCollector is not currently working after upgrading the HttpClient Library.

      In order to reproduce the test, I've created a simple CGI in Perl. The output is a plain text with a number that increases every time the request is executed:

      [agalue@centos7srv daemon]$ curl -v  http://192.168.32.106/cgi-bin/count.cgi
      * About to connect() to 192.168.32.106 port 80 (#0)
      *   Trying 192.168.32.106...
      * Connected to 192.168.32.106 (192.168.32.106) port 80 (#0)
      > GET /cgi-bin/count.cgi HTTP/1.1
      > User-Agent: curl/7.29.0
      > Host: 192.168.32.106
      > Accept: */*
      > 
      < HTTP/1.1 200 OK
      < Date: Wed, 24 Sep 2014 20:34:20 GMT
      < Server: Apache/2.4.6 (CentOS)
      < Transfer-Encoding: chunked
      < Content-Type: text/plain; charset=UTF-8
      < 
      Document Count: 1000
      * Connection #0 to host 192.168.32.106 left intact
      

      In order to collect the data, I've added the following content to http-datacollection-config.html:

      <http-datacollection-config  
          rrdRepository="/opt/opennms/share/rrd/snmp/" >
      
        <http-collection name="doc-count">
          <rrd step="30">
            <rra>RRA:AVERAGE:0.5:1:2016</rra>
            <rra>RRA:AVERAGE:0.5:12:1488</rra>
            <rra>RRA:AVERAGE:0.5:288:366</rra>
            <rra>RRA:MAX:0.5:288:366</rra>
            <rra>RRA:MIN:0.5:288:366</rra>
          </rrd>
          <uris>
            <uri name="document-counts">
              <url path="/cgi-bin/count.cgi"
                   matches="Document\sCount:\s+([0-9]+)"
                   response-range="100-399" >
              </url>
              <attributes>
                <attrib alias="documentCount" match-group="1" type="counter32"/>
              </attributes>
            </uri>
          </uris>
        </http-collection>
      
      </http-datacollection-config>
      

      On a JUnit test, I've created the following:

      public class hHttpCollectorTest {
      
          @Before
          public void setUp() throws Exception {
              File file = new File("src/test/resources/http-datacollection-config.xml");
              Assert.assertTrue(file.exists());
              FileInputStream is = new FileInputStream(file);
              HttpCollectionConfigFactory.setInstance(new HttpCollectionConfigFactory(is));
              IOUtils.closeQuietly(is);
          }
      
          @Test
          public void test() throws Exception {
              
              CollectionAgent agent = EasyMock.createMock(CollectionAgent.class);
              EasyMock.expect(agent.getInetAddress()).andReturn(InetAddressUtils.addr("192.168.32.106"));
              EasyMock.expect(agent.getNodeId()).andReturn(1);
              EventProxy eproxy = EasyMock.createMock(EventProxy.class);
      
              Map<String, Object> parameters = new HashMap<String, Object>();
              parameters.put("collection", "doc-count");
      
              EasyMock.replay(agent, eproxy);
      
              HttpCollector c = new HttpCollector();
              CollectionSet collectionSet = c.collect(agent, eproxy, parameters);
              Assert.assertNotNull(collectionSet);
              Assert.assertEquals(ServiceCollector.COLLECTION_SUCCEEDED, collectionSet.getStatus());
      
              EasyMock.verify(agent, eproxy);
          }
      }
      

      The test fails with a similar exception I found on a test environment running 1.12.9:

      2014-09-24 16:36:22,232 DEBUG [main] UNCATEGORIZED: initialize: initializing http collection config factory.
      2014-09-24 16:36:22,262 DEBUG [main] org.exolab.castor.util.LocalConfiguration: Trying to load configuration file from file:/Users/agalue/Development/opennms/git/1.12/opennms-services/target/classes/castor.properties
      2014-09-24 16:36:22,534 DEBUG [main] org.opennms.netmgt.collectd.HttpCollector: doSubs: getHost substituted as "192.168.32.106"
      2014-09-24 16:36:22,545 INFO  [main] org.opennms.netmgt.collectd.HttpCollector: doCollection: collecting for client: org.apache.http.impl.client.DefaultHttpClient@73aecc3a using method: GET http://192.168.32.106:80/cgi-bin/count.cgi HTTP/1.1
      2014-09-24 16:36:22,711 DEBUG [main] org.apache.http.impl.conn.BasicClientConnectionManager: Get connection for route {}->http://192.168.32.106:80
      2014-09-24 16:36:22,724 DEBUG [main] org.apache.http.impl.conn.DefaultClientConnectionOperator: Connecting to 192.168.32.106:80
      2014-09-24 16:36:22,743 DEBUG [main] org.apache.http.client.protocol.RequestAddCookies: CookieSpec selected: compatibility
      2014-09-24 16:36:22,751 DEBUG [main] org.apache.http.client.protocol.RequestAuthCache: Auth cache not set in the context
      2014-09-24 16:36:22,751 DEBUG [main] org.apache.http.client.protocol.RequestTargetAuthentication: Target auth state: UNCHALLENGED
      2014-09-24 16:36:22,752 DEBUG [main] org.apache.http.client.protocol.RequestProxyAuthentication: Proxy auth state: UNCHALLENGED
      2014-09-24 16:36:22,752 DEBUG [main] org.apache.http.impl.client.DefaultHttpClient: Attempt 1 to execute request
      2014-09-24 16:36:22,752 DEBUG [main] org.apache.http.impl.conn.DefaultClientConnection: Sending request: GET /cgi-bin/count.cgi HTTP/1.1
      2014-09-24 16:36:22,753 DEBUG [main] org.apache.http.wire:  >> "GET /cgi-bin/count.cgi HTTP/1.1[\r][\n]"
      2014-09-24 16:36:22,753 DEBUG [main] org.apache.http.wire:  >> "Host: 192.168.32.106:80[\r][\n]"
      2014-09-24 16:36:22,753 DEBUG [main] org.apache.http.wire:  >> "Connection: Keep-Alive[\r][\n]"
      2014-09-24 16:36:22,754 DEBUG [main] org.apache.http.wire:  >> "[\r][\n]"
      2014-09-24 16:36:22,754 DEBUG [main] org.apache.http.headers: >> GET /cgi-bin/count.cgi HTTP/1.1
      2014-09-24 16:36:22,754 DEBUG [main] org.apache.http.headers: >> Host: 192.168.32.106:80
      2014-09-24 16:36:22,754 DEBUG [main] org.apache.http.headers: >> Connection: Keep-Alive
      2014-09-24 16:36:22,819 DEBUG [main] org.apache.http.wire:  << "HTTP/1.1 200 OK[\r][\n]"
      2014-09-24 16:36:22,822 DEBUG [main] org.apache.http.wire:  << "Date: Wed, 24 Sep 2014 20:36:22 GMT[\r][\n]"
      2014-09-24 16:36:22,822 DEBUG [main] org.apache.http.wire:  << "Server: Apache/2.4.6 (CentOS)[\r][\n]"
      2014-09-24 16:36:22,822 DEBUG [main] org.apache.http.wire:  << "Keep-Alive: timeout=5, max=100[\r][\n]"
      2014-09-24 16:36:22,822 DEBUG [main] org.apache.http.wire:  << "Connection: Keep-Alive[\r][\n]"
      2014-09-24 16:36:22,822 DEBUG [main] org.apache.http.wire:  << "Transfer-Encoding: chunked[\r][\n]"
      2014-09-24 16:36:22,822 DEBUG [main] org.apache.http.wire:  << "Content-Type: text/plain; charset=UTF-8[\r][\n]"
      2014-09-24 16:36:22,822 DEBUG [main] org.apache.http.wire:  << "[\r][\n]"
      2014-09-24 16:36:22,823 DEBUG [main] org.apache.http.impl.conn.DefaultClientConnection: Receiving response: HTTP/1.1 200 OK
      2014-09-24 16:36:22,823 DEBUG [main] org.apache.http.headers: << HTTP/1.1 200 OK
      2014-09-24 16:36:22,823 DEBUG [main] org.apache.http.headers: << Date: Wed, 24 Sep 2014 20:36:22 GMT
      2014-09-24 16:36:22,823 DEBUG [main] org.apache.http.headers: << Server: Apache/2.4.6 (CentOS)
      2014-09-24 16:36:22,823 DEBUG [main] org.apache.http.headers: << Keep-Alive: timeout=5, max=100
      2014-09-24 16:36:22,823 DEBUG [main] org.apache.http.headers: << Connection: Keep-Alive
      2014-09-24 16:36:22,823 DEBUG [main] org.apache.http.headers: << Transfer-Encoding: chunked
      2014-09-24 16:36:22,823 DEBUG [main] org.apache.http.headers: << Content-Type: text/plain; charset=UTF-8
      2014-09-24 16:36:22,828 DEBUG [main] org.apache.http.impl.client.DefaultHttpClient: Connection can be kept alive for 5000 MILLISECONDS
      2014-09-24 16:36:22,837 DEBUG [main] org.apache.http.wire:  << "15[\r][\n]"
      2014-09-24 16:36:22,837 DEBUG [main] org.apache.http.wire:  << "Document Count: 1100[\n]"
      2014-09-24 16:36:22,837 DEBUG [main] org.apache.http.wire:  << "[\r][\n]"
      2014-09-24 16:36:22,837 DEBUG [main] org.apache.http.wire:  << "0[\r][\n]"
      2014-09-24 16:36:22,838 DEBUG [main] org.apache.http.wire:  << "[\r][\n]"
      2014-09-24 16:36:22,838 DEBUG [main] org.apache.http.impl.conn.BasicClientConnectionManager: Releasing connection org.apache.http.impl.conn.ManagedClientConnectionImpl@3582c132
      2014-09-24 16:36:22,838 DEBUG [main] org.apache.http.impl.conn.BasicClientConnectionManager: Connection can be kept alive for 5000 MILLISECONDS
      2014-09-24 16:36:22,838 DEBUG [main] org.opennms.netmgt.collectd.HttpCollector: doCollection: Trying to devise response's locale from Content-Language header.
      2014-09-24 16:36:22,838 WARN  [main] org.opennms.netmgt.collectd.HttpCollector: doCollection: More than 1 Content-Language headers received. Ignoring them!
      2014-09-24 16:36:22,838 DEBUG [main] org.opennms.netmgt.collectd.HttpCollector: processResponse:
      2014-09-24 16:36:22,838 DEBUG [main] org.opennms.netmgt.collectd.HttpCollector: responseBody = Document Count: 1100
      
      2014-09-24 16:36:22,838 DEBUG [main] org.opennms.netmgt.collectd.HttpCollector: getmatches = Document\sCount:\s+([0-9]+)
      2014-09-24 16:36:22,838 DEBUG [main] org.opennms.netmgt.collectd.HttpCollector: flags = 0
      2014-09-24 16:36:22,838 DEBUG [main] org.opennms.netmgt.collectd.HttpCollector: processResponse: found matching attributes: false
      2014-09-24 16:36:22,838 WARN  [main] org.opennms.netmgt.collectd.HttpCollector: doCollection: no attributes defined by the response: Document Count: 1100
      2014-09-24 16:36:22,839 DEBUG [main] org.apache.http.impl.conn.DefaultClientConnection: Connection 0.0.0.0:52474<->192.168.32.106:80 closed
      2014-09-24 16:36:22,839 ERROR [main] org.opennms.netmgt.collectd.HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: Unexpected exception caught during HTTP collection: No attributes specified were found: : client URL: 
      org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: Unexpected exception caught during HTTP collection: No attributes specified were found: : client URL: 
      	at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:329)
      	at org.opennms.netmgt.collectd.HttpCollector.access$0(HttpCollector.java:285)
      	at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:195)
      	at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:148)
      	at PlayWithHttpCollector.test(PlayWithHttpCollector.java:44)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
      	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
      	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
      	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
      	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
      	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
      

      It seems like it is getting the response from the remote server, but the HttpCollector code is doing something wrong (for the new version of HttpClient) with the response (I think).

        Attachments

          Activity

            People

            • Assignee:
              agalue Alejandro Galue
              Reporter:
              agalue Alejandro Galue
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: