S3Upload microservice uploads all the files to a specified bucket.

Icon

Get the Access Key id and Secret Key from the AWS Management Console at https://console.aws.amazon.com/s3.

Icon

Provide AmazonS3FullAccess policy to the user to get the full access to the S3 service.


Configuration and Testing

Component Configuration

The figure below illustrates the Component Property Sheet  (CPS) panel with Expert Properties view enabled.


Figure 1: Component Configuration properties for S3Upload

Process Message Based on Property

The property helps components to skip certain messages from processing.

Icon

Refer the Process Message Based On a Property section under the Common Configurations topic.

Error handling configuration

The remedial actions to be taken when a particular error occurs can be configured using this attribute. 

Click the ellipsis button against this property to configure Error Handling properties for different types of Errors. By default, the options Log to error logs, Stop service and Send to error port are enabled.

Refer Error Handling section in Common Configurations for detailed information.

AWS Connection Details

Click the AWS Connection Details ellipsis  button to configure the properties.


Figure 2: AWS Connection properties

Access Key

Specify the Access Key id

Secret Key

Specify the Secret Key

Region

The region of AWS Management console.

Chunk Size

Chunk Size should be more than or equal to 5 MB. If the file size is more than 5MB, it is uploaded as parts of specified chunk sizes. If the provided file is less than 5MB, it will be uploaded as a single chunk.

Bucket Name

Specify the S3 bucket name. As the component does not create the bucket, specify the existing one only.

Server Side Encryption ?

Select this option for the server to encrypt the data before storing in the bucket.

Server  Encryption Mode

This option appears only when Server Side Encryption ? property is selected. In this case,  Amazon S3 manages the keys required for encrypting  by itself.

Storage Mode

The current API provides three modes of Storage classes.

  • STANDARDDefault storage class; this storage class is ideal for performance-sensitive use cases and frequently accessed data.
  • Standard Infrequent Access (Standard - IA)Amazon S3 Standard - Infrequent Access (Standard - IA) is an Amazon S3 storage class for data that is accessed less frequently, but requires rapid access when needed.
  • Reduced RedundancyThe Reduced Redundancy Storage (RRS) storage class is designed for noncritical, reproducible data stored at lower levels of redundancy than the STANDARD storage class, which reduces storage costs.

File Name Prefix

The specified prefix is prepended to the  uploaded file.

Correlation Id Property

Specify the Correlation Id message property name. The input messages will be grouped based on the correlation Id property. Messages which has the same property value will be treated as a single file. This value is appended to the FileNamePrefix property, if specified, and treated as the file name.

Completeness Condition Property Name

Name of the Completeness Condition Property. Component treats that the file was completed on receiving a message with completeness property equals to completeness property value.

Completeness Condition Property Value

Value of the Completeness Condition property. Component treats that the file was completed on receiving a message with completeness property equals to completeness property value.

Bucket Name Property

Specifies the Bucket name property name. Bucket name can be provided from the input whose value takes precedence over the one provided in CPS.

Sequence Number Property

This property is used to identify duplicate sequence numbers; discards the duplicate input arrived at its input port.

Is Input Binary

Specifies whether the input messages are byte messages or not.

Is Text Base64 Encoded

Specifies if the input text needs decryption using Base64.

Threadpool Configuration

This property is used when there is a need to process messages in parallel within the component, still maintaining the sequence from the external perspective. 

Icon
  • This needs to be used only in such circumstances.
  • If sequential processing is not required, please use sessions on the input port.

Click the Threadpool Configuration ellipsis  button to configure the Threadpool Configuration properties. 


Figure 3: Threadpool Configuration properties

Enable Thread Pool

Enable this option to configure the properties that appear as below.

Pool Size

Number of requests to be processed in parallel within the component. Default value is '1'.

Batch Eviction Interval (in ms)

Time in milliseconds after which the threads are evicted in case of inactivity. New threads are created in place of evicted threads when new requests are received. Default value is '1000'.

Functional Demonstration

Uploading a file into the AWT Management console and displaying the status. Configure S3Uploader as described in Configuration and testing section and use feeder and display component to send sample input and check the response respectively.


Figure 4: Demonstrating a scenario with sample input and output

Input Message


Figure 5: Input XML sent using feeder

Output Message


Figure 6: Output demonstrating the status of the uploading file

The AWS Console displays the detailed state of storage class (Reduced Redundancy in the case of test34.pdf).

Encryption status used to store the data.

Adaptavist ThemeBuilder EngineAtlassian Confluence