HTTPAdapters
The HTTPAdapter component enables the user to get content from an external HTTP server (web server).
Hyper Text Transfer Protocol (HTTP) is one of the most widely used protocols on the Internet today. Various event processes are being hosted on the World Wide Web, due to which event process developers face the challenge of integrating their existing solutions to work with HTTP. Fiorano HTTP components help event process developers achieve this task with minimum knowledge of this protocol.
Fiorano HTTP component enables event processes to use either the Get or the Post method. This component can function in conjunction with other Fiorano components to solve business problems in a highly productive manner.
The GET implementation
Consider a stock portfolio management company that employs several consultants who advise clients on investing their money. The Shares and Scripts section consists of three clerks who track the stock quotes in three different verticals: IT, Automobiles, and Pharmaceuticals. The task of sending requests for stock quotes can be accomplished by running multiple instances of the Get implementation. Each instance constantly provides information for a different stock quote. The results can then be sent to other components for further processing.
The POST implementation
Consider an enterprise implementing an Enterprise Resource Planning (ERP) system to automate its inventory management system across its suppliers and customers. Suppliers are generally not in the vicinity of the manufacturing facility and hence it is not possible to have a dedicated link between the two parties. In this case, the supplier can provide web access to the enterprise through a secure interface and the Post method can be used to post purchase order information at the supplier server.
Points to note
- The component never tries to guess the content type like most browsers.
- If the HTTP server is secured using Basic or Digest authentication and required details are not provided in the component CPS, a runtime UI is displayed to capture the same.
Configuration and Testing
Managed Connection Factory
Connection details are configured in the first panel of the Configuration Property Sheet (CPS), Managed Connection Factory (MCF). The figure below illustrates the panel with Expert Properties view enabled.
Figure 1: Managed Connection Factory in HTTPAdapters MCF
Connection Properties
Use Connection Details From Input
If enabled, it will use the connection details from the input request. If the Connection property is not specified in input, then the value provided in this panel will be used.
Connection Configuration
Click the ellipsis 
button to set up connection configuration with the attributes as below.
Figure 2: Connection Configuration details in MCF panel
HTTP Host Name/IP
The host name or the IP address where the web server is located. The URI relative to this host name/ IP has to be configured in the request details editor in Interaction configurations. It can also be sent in the URI element of the input message.
HTTP Port
The port number on which web server is running .
HTTP Request Timeout
This Specifies the time for which this component will wait for an HTTP request to be acknowledged. Default value is 0, specifies infinite timeout.
Allow Auto Redirection
Determines the action to be taken if the requested page sends the request to another page.
If enabled, it allows automatic redirection of the requested page if required.
Force HTTP1.0
If enabled, HTTP1.0 protocol is used while sending requests to the web server and if disabled, HTTP1.1 protocol would be used while sending requests.
Proxy Settings
Please refer Proxy Settings section in Common Configurations page.
Connection Pool Params
Please refer Connection Pool Params section in Common Configurations page.
Client Host
Name of the Host machine from which the authentication request is originated from. This attribute is optional.
Proxy Settings
HTTP Authentication
Click the ellipsis button to provide details of HTTP authentication.
Use HTTP Authentication
Enables this property to specify authentication information, if any. Do not enable this when the requested resource does not require any authentication.
Two types of authentications are Basic and Digest, which are explained below.
Basic
A method designed to allow client program to provide credentials in the form of user name and password while making a request. These credentials are passed in plain text format using Base64 encoding. When you choose Basic type, provide the following credentials.
- Username
The name of the user with privileges to access the protected resource. - Password
The password for the user name specified above.
Digest
In this method, request is encrypted using MD5 cryptographic hashing and sent to web server. When you choose Basic type, provide HTTP host name, port and domain address in the respective fields, apart from username and password.
SSL Security
Please refer SSL Security section in Common Configurations page.
ByPass Proxy for Hosts
Semi-colon separated list of hostnames/ IP Addresses for which the request must be sent without passing the proxy specified, if any. If no proxy is defined, this property is ignored.
Interaction configuration
Business logic configuration details are configured in the second panel, Interaction Configurations. Figure below illustrates the panel with Expert Properties view enabled.
Figure 3: Business logic configuration in Interaction Configurations panel
Attributes
HTTP Request Property
This property defines properties of the request that is sent to web server.
Click the ellipsis button to launch an editor for providing these configurations.
Figure 4: Editor for configuring HTTP Request properties (properties for GET method)
Request Details
Method
The HTTP method that is used to send the request to the server. This is either set to GET or POST.
The implementation and usage of these methods are described in the sections: The GET implementation and The POST implementation respectively.
Request Properties depend on the HTTP method specified here. Above figure shows the request properties when HTTP method is set to GET.
GET
Request properties when HTTP method is set to GET are shown in the above figure and described as below.
URI
The Uniform Resource Identifier of the resource that is being requested by the component. This will be calculated relative to the host name/IP and port provided in the Managed Connection Factory.
Cookie
HTTP cookies provide the server with a mechanism to store and retrieve state information on the client application's system. This mechanism allows Web-based applications the ability to store information about selected items, user preferences, registration information, and other information that can be retrieved later.
The cookie can be specified in the form of name-value pairs separated by semicolon, for example, name1=value1;name2=value2. The input schema will have an element 'Cookie' corresponding to this property.
Pragma
The Pragma general-header field is used to include implementation- specific directives that might apply to any recipient along the request/response chain. For more information, refer section 14.32 at http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
The input schema will have an element 'Pragma' corresponding to this property.
Custom Headers
Use Custom Headers
If there are any custom headers to be sent along with the request, then enable this property.
If custom headers are enabled, any custom headers specified in the table are added as child elements of this Custom Headers element. Headers can be added using the editable table UI as shown in the figure below.
Figure 5: Custom Headers configuration
POST
When HTTP method is set POST, request properties are as shown in the figure below.
Figure 6: Request properties when HTTP method is set to POST
Content Encoding
The Content-Encoding header field is used as a modifier to the media-type. When present, its value indicates what additional content codings have been applied to the body, and thus what decoding mechanisms must be applied in order to obtain the media-type referenced by the Content-Type header field.
Content-Encoding is primarily used to allow a document to be compressed without losing the identity of its underlying media type.
An element Content-Encoding will be added corresponding to this property in the input schema of the component.
Content Label
Content Label is used to specify labels for controls that do not have implicit labels.
Content Type
The Content-Type header field indicates the media type of the body sent to the recipient. The component supports following values:
- text/html
- image/jpeg
- model/vrml
- video/quickline
- application/java
- text/xml
- text/css
- text/javascript
- multipart/form-data
An element Content-Type will be added corresponding to this property in the input schema of the component. To upload files, set Content-Type to multipart/form-data. When the Content-Type is set to multipart/form-data, then the input schema does not contain element Content-Type.
Custom Headers
Please refer the respective subsection under GET section above.
Parameter/Post Data Details
When HTTP method is set to POST, then the post data type can be configured using the below attributes:
Figure 7: Parameters/PostData configuration
Simple Text
This is used when data to be posted is simple text. The text to be posted can be specified by using the text area that opens by clicking the button Post Data. In this case, an element text is added as a child to entity element in the schema of the input port corresponding to the text that is being posted.
Binary Type
This is used when data to be posted is in binary format. The content can be specified by clicking on the Load from File button. In this case, an element BinaryData is added as child to entity element in the schema of the input port corresponding to the data that is being posted.
Figure 8: Panel showing Binary Type Parameters/Post Data Details
XML Type
This is used when data to be posted is in the form of XML conforming to a specific schema. The content can be specified in the schema editor; an element XMLData is added as child to entity element in the schema of the input port, whose type is same as the schema provided.
Figure 9: Panel showing XMLType Parameters/Post Data Details
Parameters
This is used when parameters are being posted. The schema of the component has an element entity which accepts parameters as name value pairs.
Figure 10: Panel showing Parameters option in Parameters/Post Data Details
Apply URL Encoding on parameters
If this property is selected, parameters are converted to the application/x-www-form-urlencoded MIME format (HTML form encoding) using the platform's default encoding scheme.
File Parameters Details
When HTTP method is set to POST and the Content-Type is set to multipart/form-data, then the files need to be post can be configured here as shown in the figure below. This button is enable when the Content-Type is set to multipart/form-data
Figure 11: File Parameters Details (POST method, Content-Type set to 'multipart/form-data')
Example
To upload a file at http://www.filefactory.com/upload.php, the following parameters should set
- redirect to 1
- enabled to 1
- file to the path of the file which has to be uploaded
- uploadStart to true
- uploadOne to true
Parameters redirect, enabled, uploadStart, uploadOne should be given under Parameters and the parameter file should set under Files.
Refer to section Input and Output for details about the effects of these configurations on input and output structures.
HTTP Response Property
When a request is sent to HTTP server, a HTTP response is obtained. Output message is created from this HTTP Response based on the configuration details of this HTTPResponse property.
Click the ellipsis button to launch an editor for providing these configurations.
Figure 12: HTTPResponse Property configuration
Response Code
The HTTP response code that is sent from the server. A response code will be included in the output message if this property is enabled. If the Response Body (explained below) property is chosen as XML, an element Response-Code is added to the schema of the output port. Otherwise, a message Header with name HTTP_Response-Code is set as message property on output message.
Response Message
The message corresponding to the response code returned. If the corresponding checkbox is selected, it is included in the output message. If the Response Body property is chosen as XML, an element Response-Message is added to the schema of the output port. Otherwise a message Header with name HTTP_Response-Message is set as message property on output message.
Headers form part of the HTTP response stream which carries information about the response stream that is sent. The headers below are standard HTTP response headers. If the corresponding checkbox is selected, it is included in the output message. An element with the same name as the header is added as a child element of Header element in the schema of the output port if the Response Body property is chosen as XML. Otherwise a message header with a name prefixed with HTTP_ is set as message property on output message.
Content-Type
The Content-Type header field indicates the media type of the body sent to the recipient.
Content-Length
The Content-Length header field indicates the size of the body sent to the server. Any Content-Length greater than or equal to zero is a valid value.
Content-Location
The Content-Location header field MAY be used to supply the resource location for the entity enclosed in the message when that entity is accessible from a location separate from the requested resource's URI.
Content-Range
The Content-Range header is sent with a partial body to specify where in the full body the partial body should be applied.
Content-Encoding
The Content-Encoding header field is used as a modifier to the media-type. When present, its value indicates what additional content codings have been applied to the body, and thus what decoding mechanisms must be applied in order to obtain the media-type referenced by the Content-Type header field.
Set-Cookie
The Set-Cookie header is sent by the server in response to an HTTP request, which is used to create a cookie on the user's system. The Cookie header is included by the client application with an HTTP request sent to a server, if there is a cookie that has a matching domain and path.
The Set-Cookie value can be specified in the following format name1=value1;name2=value2.
Last-Modified
The Last-Modified header field indicates the date and time at which the origin server believes the variant was last modified. The exact meaning of this header field depends on the implementation of the origin server and the nature of the original resource. For files, it may be just the file system last-modified time. For entities with dynamically included parts, it may be the most recent of the set of last-modify times for its component parts. For database gateways, it may be the last-update time stamp of the record. For virtual objects, it may be the last time the internal state changed.
Please refer to http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html for more detailed information on standard HTTP headers.
Response Charset
Response charset is the charset used to convert bytes data to characters received in the Response message
By default, the Response Charset value in the CPS will be empty. The order of priority by which the charset property is considered is as below:
- If the server sends the charset in the header property, value for the charset is taken from the server and the "Response Charset" configured in the CPS is ignored.
- If not provided in the request by the server, the CPS property gets the priority.
- If charset is not provided in the request and not set in CPS, then default charset is set to ISO-8859-1.
Headers to be included(comma-separated)
If there are any custom headers apart from the standard HTTP headers, those headers can be specified here in a comma-separated list.
If the custom headers are added and the response type is chosen as XML, the specified properties appear as child elements of Custom-Headers element in the output message. If response type is chosen as Headers, then a message header with a name prefixed with HTTP_ will be sent in the output message.
Response Body
This property defines the format of the output message that is sent to the output port. At runtime, the component converts the HTTP response obtained from HTTP server into output message based on this property.
Headers
This property is used to set the selected headers (both standard HTTP and custom headers) as message properties on output message. All the selected headers are read from HTTP response obtained and set on output message. The response message from HTTP response is set to body of the output message.
XML
When this property is used, all the selected headers are set as elements in output schema created by the component.
The message body of the output message that is generated from HTTP response can be specified in schema editor. This property is optional. If specified, output schema depends on the response type (Headers or XML). If response type is Headers, the schema specified here is set as output schema. If response type is XML, a response message is created in the form of XML data conforming to the schema specified here and set as CDATA in the body of the output message.
Please refer to section Input and Output for details about the effects of these configurations on input and output structures.
Treat Null Content-Type As
When the Content-Type header in HTTP response stream is null, the action to be taken can be specified here.
Figure 13: Configuration of null 'Content-Type'
Exception
If this property is set, an exception is sent to error port.
application/octet-stream
If this property is set, then Content-Type header of the response is set to Application/octet-stream.
Plain Text ContentTypes
By default, ContentType that contains text, json or xml are treated as plain text. Apart from these, the specified comma (,) separated content types provided in this property will also be treated as plain text. If component has to treat text/csv and application/csv as plain text requests, the this property should be populated as 'text/csv,application/csv'.
Write response to file
When this property is selected, HTTP response will be written directly to a file. It has three properties associated with it, namely -
Output File Directory: This attribute requires absolute path.If not provided the file is saved in the present working directory.
Output File Name: Name of the file (extension not required).
Action if same file exists: This includes two options:
- Overwrite: Replaces the older file with the new one in case the file names are the same.
- Rename: Renames the file if a file with the same name exists. Uses properties like Append timestamp, Append date-time format postfix, Append counter.
Scheduler Configurations
Please refer Scheduler Configurations section in Common Configurations page.
Transport Configurations
Please refer Transport Configurations section in Common Configurations page.
Error Handling
Please refer Error Handling section in Common Configurations page.
Input and output
Input
The input schema for the component is defined based on the configuration of HTTPRequestProperty.
When the HTTP method is set to GET, the input schema varies based on the configuration of headers. The configuration of parameters does not effect the input schema generated.
Input schema when no headers are selected and sample input are as below.
Figure 14: Schema when there are no headers.
Figure 15: Sample input XML for schema in Figure 13
When custom headers are added as shown in the figure below, the input schema and a sample input for the same appear as in the subsequent figures.
Figure 16: Sample Headers configuration
Figure 17: Schema when custom headers are selected
Figure 18: Sample input XML for schema in Figure 17
When the HTTP method is set to Post, the input schema varies based on the configuration of headers and parameters/post data details.
When no custom headers are added and Parameters/PostData Details is set to Simple Text, the corresponding input schema and sample input appear as in the subsequent figures below.
Figure 19: Schema when parameters/postdata set to 'SimpleText'
Figure 20: Sample input XML for schema in Figure 19
When custom headers are added as shown in Figure 16 and Parameters/PostData Details is set to Binary Data, the corresponding input schema and a sample input are shown below.
Figure 21: Schema when parameters/postdata set to 'BinaryData'
Figure 22: Sample input XML for schema in Figure 21
When custom headers are added as shown in Figure 16 and Parameters/PostData Details is set to XML, the corresponding input schema and a sample input are shown below.
Figure 23: Schema when parameters/postdata set to 'XML'
Figure 24: Sample input XML for schema in Figure 20
A sample schema is provided by clicking PostData button. This schema is set as child element under XMLData element. In the input message, xml data conforming to the schema specified should be set else an error can be thrown as this is a required field.
When custom headers are added as shown in Figure 16 and Parameters/PostData Details is set to parameters, the corresponding input schema and a sample input are shown below.
Figure 25: Schema when parameters/postdata set to 'Parameters'.
Figure 26: Sample input XML for schema in Figure 25.
When custom headers are added, they are added as child elements under CustomHeaders element. The addition of parameter does not effect the input schema generated.
When Content-Type is set as multipart/form-data and the custom headers are added then the input schema and sample input appears as shown in the below figures. The Schema does not contain the Content-Type element in this case.
Figure 27: Schema when Content-Type set to 'multipart/form-data'
Figure 28: Sample XML for schema in Figure 27
The files that need to be uploaded can be given as child elements of element Files as shown above. In the element File the Name attribute specifies the name of the parameter and the Value specifies the path of the file which has to be uploaded. If the custom headers are not configured, then the element CustomHeaders will not be present in the schema.
In all the above cases except when PostData is set to XML,all the elements present in the Input message are optional. A Sample input message with no fields can be used as shown in Figure 27.
If there are no fields as shown in the above, the default values which are configured in CPS are set on the request message. Else the value of the corresponding field is taken from input message.
Figure 29: Schema when Write response to file is selected.
Figure 30: Sample XML for schema in Figure 29
Output
The output schema for the component is based on the configuration details provided for HTTPResponseProperty.
When the Response Body for output message is set as Headers, no schema is set on the output port. If any headers are selected or if any custom headers are configured, then they are set as message properties on the output message.
When the Response Body for output message is set as XML, output schema generated depends on the schema provided by clicking the Response button.
When no schema is provided in Response, the corresponding output schema and sample output XML are as below.
Figure 31: Output Schema when no schema is set in Response.
Figure 32: Sample XML for schema in Figure 29.
If any custom headers are configured, then they are added as child elements under CustomHeaders element which is set as child element under Header element.
When a schema is provided in Response, the corresponding output schema, and sample output XML are as below.
Figure 33: Output Schema when sample schema is set in Response
Figure 34: Sample XML for schema in Figure 33
Figure 35: Output Schema when Response Body is set to Headers
Figure 36: Sample XML for schema in Figure 33
Figure 37: Output Schema when Response Body is set to XML
Figure 38: Sample XML for schema in Figure 37
SSL Setup
To communicate with https websites, SSL has to be enabled in HTTPAdapter and the values for KeyStore, TrustStore, KeyStore Password, TrustStore password and KeyStore Client Key have to be specified in the SSL settings dialog in the CPS.
Client keystore and truststore are required for SSL communication. The procedure to obtain the certificate from the https website and to generate the client keystore and truststore is explained in SSL Configuration – Client perspective section.
Using Keystore and Truststore in an SSL Application section explains the procedure to use the generated Client keystore and truststore in HTTPAdapters for SSL configuration.
The default HTTP Port to be used for SSL communication is 443. If the HTTP server uses a different port then it has to be mentioned.
Functional demonstration
Scenario 1
Send a request to a Web server and display the response.
Configure the HTTPAdapters as described in Configuration and Testing section and use feeder and display component to send sample input and check the response respectively.
Figure 39: Demonstrating Scenario 1 with sample input and output.
Input Message
Output Message
In Order Entry sample, a user is provided a web based interface to send a purchase order to a company. In case the order is accepted, HTTPAdapters is used to POST the order delivery request to a third party vendor.Use Case Scenario
Figure 40: Order Entry
Useful Tips
- When connecting to secured sites, the component accesses truststore from different locations in different launch modes:
- Separate Process - %JAVA_HOME%\jre\lib\security\cacerts
- In-memory - %FIORANO_HOME%\esb\server\profiles\certs\jssecacerts
- 'jssecacerts' in %FIORANO_HOME% does not have trusted certificates required by most of the secured sites. Hence, to connect to secured sites when component is launched in-memory, either the trusted certificates should be added to 'jssecacerts' or the property 'javax.net.ssl.trustStore' (in %FIORANO_HOME%\esb\server\bin\server.conf, if FPS is launched using server.bat/server.sh and %FIORANO_HOME%\esb\fps\bin\fps.conf, if FPS is launched using fps.bat/fps.sh) should be set to %JAVA_HOME%\jre\lib\security\cacerts.
To receive Axis Level HTTP logs along with component logs, the System Properties HTTPClient.log.file and HTTPClient.log.enable have to be set in Runtime arguments of the component.