XmlCollector enhancements

Description

It would be nice if the XmlCollector could support more than HTTP Get requests. Also, another enhancement would be to support compressed input.

Acceptance / Success Criteria

None

Lucidchart Diagrams

Activity

Alejandro Galue July 10, 2013 at 2:40 PM

All the improvements where committed on revision 4a070a6ba61f27b6fdb57a7e5a58fbe8eab91dde for 1.12

1) Handling Port

There is no need to define the port as a parameter on collectd-configuration.xml, as you can define it as part of the URL on the xml-source tag:

2) Custom HTTP headers

There is no need to define custom headers like VirtualHost as a parameter on collectd-configuration.xml, as it is more flexible to use the new request object for that.

3) Custom HTTP Client Parameters

There is no need to define custom parameters like timeout or retries as parameters on collectd-configuration.xml, as it is more flexible to use the new request object for that.

4) The new Request Object

Now you can define the payload with custom data inside the xml-source tag to personalize the HTTP request you want to perform. This feature can be reused by other kind of protocols in the future.

4.1) GET Request with custom headers

4.2) POST Request with XML Payload

4.3) POST Request with JSON Payload

4.4) POST Request with Form Data

5) Pending

I don't understand the use case for "priortime" and "currenttime", maybe some examples will help, but I appreciate doing that on a separate Jira Issue.

VERY IMPORTANT:

  • Always put the payload inside a CDATA in order to avoid problems when parsing the configuration file.

  • You can use placeholders on the headers, parameters and content.

  • Remember to set the correct content-type, specially for sending forms.

  • For JSON data, please include spaces between each curly brackets (or braces) to avoid confusions with the placeholders.

Alejandro Galue July 9, 2013 at 3:30 PM

Added basic HTTPS support on revision 85509ec13406337a1f5c95b450cba597359af7c6 for 1.12

Ron Roskens February 6, 2013 at 7:51 PM

For XML and JSON, I think that looks good.

For FORM, I think it should follow the HTTP standard and use the ampersand to split them.

I suppose you could also do something with the jaxb definition and allow either a <content /> block or multiple <form-field /> tags.

then treat it similar to the buildRequestParameters() method in HttpCollector?

For placeholder parameters, Peter's comments show the need for {priortime} and {currenttime}, with some special handling for mod'ing the time so they sync up with step. (time % step) So maybe also {priortime%step} and {currenttime%step}?

How would you handle a virtual website too? Would we allow a <http-header key='Host' value='{vhost}' />? Can we map {somename} to a <parameter> entry on the service?

Alejandro Galue February 6, 2013 at 9:47 AM

I'm wondering how can we define a POST request from the configuration perspective. Maybe we can define something like this:

To send an XML:

In order to define the internal XML to be sent, we must declare it using CDATA, otherwise, the JAXB Parser is going to try to find a Java Object that represents the XML, and as this can be dynamic, it is better to do it that way, otherwise the XML Collector won't be usable

To send a JSON String:

To send a FORM:

I was thinking on using ";" as a field separator. Maybe the field values can use quotes, or we can put an entry per line.

Of course, for all cases, we can use the placeholders in the same way we use it for the URL.

Thoughts?

Peter Nixon August 31, 2012 at 12:55 PM

I am definitely +1 for HTTP POST support.

I need to POST to an XML query containing datetime range a to a stats interface like so:

wget --post-data "<QueryOmStats><StartTime>2012-08-29T14:15:00Z</StartTime><EndTime>2012-08-29T14:30:00Z</EndTime><DiameterPcefStats /></QueryOmStats>" qOhttp://10.125.142.30/mi/xmlInterfaceRequest.do?user=USER&pwd=PASSWORD

Fixed

Details

Assignee

Reporter

Components

Fix versions

Affects versions

Priority

PagerDuty

Created July 30, 2012 at 11:41 AM
Updated January 27, 2017 at 4:19 PM
Resolved July 10, 2013 at 2:40 PM