Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

HTTP Collector throwing exception "Host must be set to create a host URL"

Description

The following stack trace is being logged in collectd.log when attempting to use a HTTP Collector.

2011-02-17 18:24:38,361 ERROR [CollectdScheduler-50 Pool-fiber2] CollectableService: An undeclared throwable was caught during data collection for interface <ip-address>/Apache-Stats
org.opennms.netmgt.collectd.CollectionException: An undeclared throwable was caught during data collection for interface <ip-address>/Apache-Stats
at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:408)
at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:314)
at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:295)
at org.opennms.core.concurrent.RunnableConsumerThreadPool$FiberThreadImpl.run(RunnableConsumerThreadPool.java:427)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.lang.IllegalStateException: Host must be set to create a host URL
at org.apache.commons.httpclient.HostConfiguration.getHostURL(HostConfiguration.java:316)
at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:466)
at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:98)
at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:173)
at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:132)
at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:277)
at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:380)
... 4 more

I have replaced the IP address with <ip-address> in this bug for privacy. Below is an excerpt from http-datacollection-config.xml

<http-collection name="Apache-Stats">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:8928</rra>
<rra>RRA:AVERAGE:0.5:12:8784</rra>
<rra>RRA:MIN:0.5:12:8784</rra>
<rra>RRA:MAX:0.5:12:8784</rra>
</rrd>
<uris>
<uri name="apache">
<url path="/server-status/?auto"
user-agent="Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/412 (KHTML, like Gecko) Safari/412"
matches="(?s).?Total\sAccesses:\s([0-9]+).?Total\skBytes:\s([0-9]).*?CPULoad:\s([0-9\.]).?Uptime:\s([0-9]+).?ReqPerSec:\s([0-9\.]).*?BytesPerSec:\s([0-9\.]).?BytesPerReq:\s([0-9\.]+).?BusyWorkers:\s([0-9]).*?IdleWorkers:\s([0-9]).*" response-range="100-399" >
</url>
<attributes>
<attrib alias="TotalAccesses" match-group="1" type="gauge32"/>
<attrib alias="TotalkBytes" match-group="2" type="gauge32"/>
<attrib alias="CPULoad" match-group="3" type="gauge32"/>
<attrib alias="Uptime" match-group="4" type="gauge32"/>
<attrib alias="ReqPerSec" match-group="5" type="gauge32"/>
<attrib alias="BytesPerSec" match-group="6" type="gauge32"/>
<attrib alias="BytesPerReq" match-group="7" type="gauge32"/>
<attrib alias="BusyWorkers" match-group="8" type="gauge32"/>
<attrib alias="IdleWorkers" match-group="9" type="gauge32"/>
</attributes>
</uri>
</uris>
</http-collection>

And below is the config from capsd-configuration.xml:

<protocol-plugin protocol="Apache-Stats" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="on" user-defined="false">
<property key="port" value="80" />
<property key="timeout" value="3000" />
<property key="retry" value="2" />
<property key="url" value="/server-status/?auto" />
</protocol-plugin>

And from collectd-configuration.xml:

<service name="Apache-Stats" interval="300000" user-defined="false" status="on" >
<parameter key="http-collection" value="apache-stats" />
<parameter key="retry" value="1" />
<parameter key="timeout" value="2000" />
</service>

Thanks
Paul

Environment

CentOS 5.5

Acceptance / Success Criteria

None

Lucidchart Diagrams

Activity

Ron Roskens November 6, 2012 at 12:35 PM

I think this was fixed with commit 1f3d2e14 back on Tue Aug 3 2010.

HttpCollector.doCollection(collectionSet, collectionResource) method no longer throws a new HttpCollectorException with an HttpClient object attached to it. So when this thrown exception gets logged inside CollectableService.doCollection() it does not cause a new java.lang.IllegalStateException to be thrown.

Seth Leger April 5, 2012 at 2:24 PM

I enhanced the HttpCollectorTest in 1.11 so that it tries to collect from a system while using a malformed regex and it does not trigger this error. Instead, it throws a PatternSyntaxException which makes sense and should point users at regex problems in the config.

We still have occasional reports of this error occurring so I'm going to leave the issue open. We need to figure out how to reproduce it and add a test case for this to HttpCollectorTest.

Alejandro Galue September 29, 2011 at 7:48 AM

I tried the sample configuration on 1.9.92-SNAPSHOT and 1.8.15-SNAPSHOT, and it works for me on both.

Here are the details (I'm using a provisioned node with the same configuration on both instances of OpenNMS):

Here is part of the logs from 1.8.15-SNAPSHOT:

The relevant part of the logs on 1.9.92-SNAPSHOT is very similar.

Is there any additional stuff you want me to try ?

Tarus Balog September 28, 2011 at 5:27 AM
Edited

Actually, the issue is that if the is a failure in the regex, the collector will throw the error:

2011-09-28 11:20:12,651 DEBUG [CollectdScheduler-50 Pool-fiber2] HttpCollector: processResponse: found matching attributes: false
2011-09-28 11:20:12,651 ERROR [CollectdScheduler-50 Pool-fiber2] CollectableService: An undeclared throwable was caught during data collection for interface 212.218.252.54/OpenNMS-Login
org.opennms.netmgt.collectd.CollectionException: An undeclared throwable was caught during data collection for interface 212.218.252.54/OpenNMS-Login
at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:414)
at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:316)
at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:295)
at org.opennms.core.concurrent.RunnableConsumerThreadPool$FiberThreadImpl.run(RunnableConsumerThreadPool.java:427)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalStateException: Host must be set to create a host URL

Since the default error level is set to WARN, you'll only see the error for the URL and not that the regex failed. I think this should be corrected.

Tarus Balog September 28, 2011 at 5:15 AM

This is definitely broken in 1.8.14. Trying to run the server-status?auto example in class and it fails with the same host must be set log.

Details

Assignee

Reporter

Labels

Components

Affects versions

Priority

PagerDuty

Created February 17, 2011 at 1:35 PM
Updated September 21, 2021 at 6:24 PM