Contents

The FTP Get component is used for downloading files from the FTP Server. It can be used for downloading a single file or all files in a directory to the desired location.

Using the FTP Get component, a file can be downloaded by any of the following methods:

  • By monitoring a remote directory for any modifications such as file addition or updation and download of the corresponding file to the desired location.
  • By downloading the file specified in the input message to the desired location.

The FTP Get component uses the FTP protocol for file transmission. The component ensures uninterrupted download by attempting to reconnect to the remote server in case the connection to the server is lost.

Configuration and Testing

Managed Connection Factory

The connection properties can be configured using the properties ofManagedConnection Factory panel.


Figure 1: Managed Connection Factory panel

Connection Properties

Use Connection Details From Input

Parameters to create the connection can be specified in the input message when this property is enabled. If this property is selected the validation errors in the managed connection factory panel of the CPS are treated as warnings. So the user can bypass this step without giving valid configuration and complete the configuration of the component. If valid properties are not provided even in the input message exception will be thrown at runtime.

Connection Configuration

Protocol

Select the Protocol from any of the following options available:

  • FTP: Select this option if you want the protocol to use as File Transfer Protocol.
  • SFTP: Select this option if you want the protocol to use as Secured File Transfer Protocol. If we select this protocol, then we need to provide Client Authentication Type. If we select Client Authentication Type as Password, then client's username and password are sufficient to log in successfully. These details can be set by using the properties Login and Password. If we select Client Authentication Type as Public Key or Both we need to provide the details of Private Key File, Key File Type and Key File Password. For detail explanation of SFTP setting, please refer Scenario 3 under section Functional Demonstrations.
  • FTPS: FTPS is an extension to the commonly used File Transfer protocol (FTP) that adds support for the Transport Layer Security(TLS) and the Secure Sockets Layer(SSL) cryptographic protocols. FTPS should not be confused with the SFTP protocol, an incompatible secure file transfer subsystem for the Secure Shell(SSH) protocol. It is also different from the Secure FTP, the practice of tunneling FTP through an SSH connection. If selected, then the property Implicit FTPS is enabled. If Implicit FTPS property is enabled, then Implicit FTPS takes SSL one step further than simply requiring that SSL- related commands must be sent first like you can with Explicit SSL; with Implicit FTPS, an SSL handshake must be negotiated before any FTP commands can be sent by the client. In addition, even though Explicit FTPS allows the client to arbitrarily decide whether to use SSL, Implicit FTPS requires that the entire FTP session must be encrypted. Basically the way Implicit FTPS works is that an FTP client connects to the command/control channel, in this case using port 990 instead of using port 21 as in the caseofFTPS, and immediately performs an SSL handshake; after SSL has been negotiated, additional FTP commands for the session can be sent by the FTP client.
FTP Protocol Settings


Figure 2: Connection Configuration

Remote Host

The hostname/IP address of the machine where the FTP server is running.

Port

The port number on which the FTP server is running.

Login

Username of the FTP user.

Password

Password of the FTP user. This field will be disabled if a user selects Protocol as SFTP and Client Authentication Type as Private Key.

SFTP Protocol Settings

All properties except the below ones remain the same.

Client Authentication Type

This property determines the authentication type for the client validation for the specific protocol.


Figure 3: Client Authentication Type options

  • Password: If selected, the client's username and password are used for client authentication.
  • Private Key: If selected, the client's Private Key and the Key File Password are used for client's authentication. (Key file password is different from client password). This option is available only in SFTP protocol.
  • Both: This option authenticates using a private/public key-pair, followed by password authentication. If the authentication fails while using client's private key, then it will try to authenticate using password authentication. This option is available only in SFTP protocol.
Private Key file

The private key file path in the local machine used for client authentication in case of protocol SFTP. The path should include the file name also. The key file should be present on the machine where the peer server (on which peer the component is running) is running. This property is visible when the Protocol is selected as SFTP and Client Authentication Type as Private Key or Both.

Private Key Type

Determines the private key type either 'DSA' or 'RSA'. This property is visible when the Protocol is selected as SFTP and Client Authentication Type as Private Key or Both.

Private Key password

The private key file's password when the property protocol is set to SFTP and the property Client Authentication Type is set to Private Key or Both.

Icon

Private Key File password is different from client's password.

FTPS Protocol Settings

All properties except the below ones remain the same.

Use Implicit FTPS

To enable implicit FTPS, when using the FTPS protocol.

Client Authentication Type


Figure 4: Client Authentication Type options

This property determines the authentication type for the client validation for the specific protocol.

  • Password: If selected, the client's username and password are used for client's authentication.
  • Password and KeyStore: This option authenticates using both password and KeyStore. The Keystore file of JKS type is used to load Private key and Certificate required to authenticate the client apart from password authentication. This option is visible only for FTP protocol.


Figure 5: Properties supporting Authentication Type 'Password and KeyStore''

Private Key Password

The private key file's password when the property protocol is set to FTPS and the property Client Authentication Type is set to Password & KeyStore.

Icon

Private Key password is different from c lient'spassword.

KeyStore file (JKS type)

The KeyStore file path in the local machine used for client authentication in case of protocol FTPS. The path should include the file name also. It should be of JKS type. The KeyStore file should be present on the machine where the peer server (on which peer the component is running) is running. This property is visible when the Protocol is selected as FTPS and Client Authentication Type as Password & KeyStore. It is used to load Private key and Certificate required to authenticate the client apart from password authentication.

KeyStorePassword

The Password for the KeyStore File given above. This property is visible when the Protocol is selected as FTPS and Client Authentication Type as Password & KeyStore.

KeyStoreAlias

The Alias name used for the Keystore file given above inthKeystore file path. This property is visible when the Protocol is selected as FTPS and Client Authentication Type as Password & KeyStore.

Proxy Settings

FTP adapters support HTTP and SOCKS proxies. HTTP is the default option. Here, you may configure the proxy server settings.


Figure 6: Proxy Settings

  • Use Proxy Server: Select this option if the connection has to be established using a proxy server.
  • Proxy Address: The IP address or the host name of the machine where the proxy server is running.
  • Port Number: Port number on which the proxy server is running.
  • Username: The user name to log into the proxy server.
  • Password: Password for the user name provided.
  • SOCKS Proxy: Enable this property to use SOCKS protocol to connect to the proxy server.

Connection Pool Params

You can specify the details for maintaining the pool of connections in the component.


Figure 7: Connection Pool Params attributes

Please refer the corresponding section in the Common Configurations page.

SSL Security

Click the SSL Security ellipsis button to launch the editor to set SSL configurations.

Enable SSL

By checking this we can enable SSL settings and we can access FTPS (FTP Over SSL) server.

Please refer the corresponding section in the Common Configurations page.

Advanced Settings


Figure 8: Advanced Settings attributes

Connect mode

You can select the type of FTP connection – Active or Passive.

Icon

This property will be ignored if you select protocol as SFTP.


Figure 9: Connect mode options

  • Active: In Active mode the FTP client specifies the data port that the FTP server is going to connect on and waits for the FTP server to connect. The IP address and port numbers are sent to the FTP server by the FTP client using the PORT command.
  • Passive: In passive mode the FTP server specifies the data port that the FTP client will connect on and waits for the FTP client to connect. The FTP client will ask the FTP server for the server's IP address and port number by issuing the PASV command to the FTP server. This will usually solve the problem of firewalls filtering the incoming data connection.

Transfer type

Specifies the transfer type.


Figure 10: Transfer Type options

  • Ascii: When you select Ascii mode the transferred data is considered to contain only ASCII formatted text. If you select Ascii transfer mode then the component is responsible for translating the format of the received text to one that is compatible with the operating system of fps(The fps on which the FTPGet component is running). Text files and files containing HTML, CSS mark-up are suitable for Ascii mode transfer.
  • Binary: When you select Binary mode of transfer the component transmits raw bytes of the file being transferred. All audio, video and image files are suitable for Binary mode transfer.

Resume Transfer

Resumesftptransfer from the point where download had stopped in case transfer is broken. Resume of the broken transfer depends on the FTP server. If the FTP server does not support this then the FTP adapter will start from the beginning otherwise it will start from where it was stopped.

Icon

Broken transfers will be resumed only when the Transfer type is Binary.

Extensions of the files to be filtered

When files of specific extension should not be downloaded from the server, the file extension has to be specified here. This property accepts comma separated list of file extensions. Example: *.zip, *.exe, *.dat.

Example: If this property is set to .exe and the user specifies to get the file named "installer.exe" in the request, then the component ignores that request.

Debug responses

When FTP responses are needed, enabling this property logs all the FTP responses to the Output Log of the component. Below figure illustrates a sample snapshot of the debug responses when some download happens.


Figure 11: FTP responses in FTPGet Output log

Timeout (in ms)

The TCP timeout in milliseconds for the sockets. Any operation which takes longer than the timeout value is killed with a java.io.InterruptedException.

Advanced details


Figure 12: Advanced Configuration dialog box

Current directory

When a user logs in to the FTP server, then the directory to which it is changed to.Allrelative  paths in the server that are computed by the FTPGet component are relative to this directory.

Example: The user's home directory is /home/user and the current directory is set to /home/user/Fiorano then when the user logs into the FTP server, the directory will be changed to /home/user/Fiorano.
If the properties Working directory, Error directory and Processed directory of the Interaction Configurations panel are left to the default values inQueue, error queue and processedQueue respectively, these directories are created under the directory specified by this property.

SITE command parameters

Site commands are sets of extended commands that can be issued by an FTP client, and they are not defined in RFC. However, they are supported by various FTP servers, and different servers usually have different supported site commands. SITE command is used by the server to provide services specific to the system. All the server administrative tasks can be performed by the SITE command.
This property accepts a semicolon separated list of SITE command parameters that have to be executed immediately after login. These parameters are server dependent.

Example: For OS/400 platform, the server-specific format of lists or names can be changed to UNIX type formats by specifying the value LISTFMT 1; NAMEFMT 1 for this property.

Use specified format for parsing Directory Listing?


Figure 13: Option to use specified format for parsing Directory Listing and Parsing format for Directory Listing

This property is used to parse the directory listing of the FTP server. For example in case of Unix the directory is listed as follows
drwxrwxr-x 3 user group 4096 2008-10-23 14:13 fioranodrwxr-xr-x 14 user group 4096 2008-12-18 14:41 Fiorano
But onWindows the directory is listed in a different format. This listing is the output from the FTP server after executing "dir" command.

  • When this property is disabled, the parsing format will be chosen depending on the Operating System on which the FTP server is running.
  • Enable this property if a specific parsing format is to be used for parsing the Directory listing returned by the FTP server.

Example: If an FTP server is running on IIS on a Windows machine and its directory listing style is set to UNIX, enable this property and set 'Parsing format for Directory Listing' to UNIX for this property.

Parsing format for Directory Listing

This property is used to determine the format to use for parsing directory listing in the FTP server. The formats supported by the component are Windows and UNIX.

  • Unix: Select to use Unix format to parse the directory listing
  • Windows: Select to use Windows format to parse the directory listing

Testing the Connection Configuration

Server connection can be tested from within the CPS by clicking on the Test button in the Managed Connection Factory panel.


Figure 14: Result of successful connection creation

Interaction Configurations

Interaction settings can be configured in the Interaction Configurations panel.


Figure 15: Interaction Configurations panel

Source Settings

Pre Processing XSL Configuration

Pre Processing XSL configuration can be used to transform request message before processing it. Click the ellipses button against the property to configure the properties.

Refer to the Pre/Post Processing XSL Configuration section under the Common Configurations page for details regarding Pre Processing XSL configuration and Post Processing XSL configuration (below).

Post Processing XSL Configuration 

Post Processing XSL configuration can be used to transform response message before sending it to the output port.

Process Message based on a Property

In certain occasions where all the batch messages in a flow are required to be processed in sequential order, enable this property to configure messages to be processed based on a specific property.

Icon

Refer the Processing Message based on a Property section to understand the working of this property.

Response type

Specifies the type of user input to the adapter. This property provides two options:

  • File
  • Data

When the request type is File, the path of the local file which is to be transferred is specified in the input.
Refer to section Input and Output for details about the effects of these configurations on input and output structures.

Send Transfer Progress Notifications

If enabled, then one more option appears as Monitor Progress interval as shown in the figure below. When large files are being downloaded from the FTP server, the progress of the transfer can be obtained by enabling this property.


Figure 16: Transfer Progress Notification option

If this property is enabled, then the FTPGet adapter will send notifications of the downloading process at regular time intervals and this time interval can be specified by the property Monitor Progress interval (in ms). The example output of this notification is shown below; notice the BytesTransferred and TotalBytes fields in the output XML.


Figure 17: Output of the Notification

Monitor Progress interval (in ms)

The time interval (in milliseconds) between any two progress notifications. This property is enabled when the property Send Transfer Progress Notification is enabled.

Validate Input

If this property is enabled, FTPGet adapter will validate the input request with the input port XSD.

Cleanup resources (excluding connection) after each document

This closes all the resources except for the connection used by the FTPGet adapter after every request. If the less processing time is more important the less memory usage, then it is recommended to disable this property and vice versa.

Delete file after transfer

Specifies if the remote file is to be deleted after it is completely downloaded.

Target Namespace

Target Namespace for the FTP request and response XML messages.

Allow File Patterns in Input Request

Enabling this will allow File patterns in the input request. File patterns support wild characters '*' and '|'. For example "*.log' will accept all files with extension as 'log'. "*.log|*sample*" will additionally accept files having sample as part of its name. The property Extension of the files to be filtered will be ignored on selection of this property.

Check for File Locks during Transfer

Allows File patterns in the Input FTP request.

Monitoring Configuration

Please refer to the Monitoring Configuration section in Common Configurations page.

Postprocessing Action

Specifies the type of action that is to be performed after the file has been processed successfully by the FTPGET component.


Figure 18: Interaction Configurations panel With Post Processing Action

This property provides three options:

  • DELETE
  • MOVE
  • NO_ACTION


    Figure 19:  With Post Processing Action MOVE
Postprocessing Directory

Specifies the name of the directory where the file is to be moved after it has been processed successfully.

Append Timestamp

Specifies whether a time-stamp needs to be appended to the name of the file after it has been processed successfully.

Timestamp Format

If the Append Timestamp property is selected, specify the format in which the date needs to be appended.

Append Counter

Specifies whether a counter value is to be appended to the time stamp.

Elements to Decrypt

Please refer to the Configuring Components for Message Encryption section in Common Configurations page.

Monitoring Settings


Figure 20: Monitoring Settings section

Monitor Directory

This property can be used to make the FTPGet adapter poll a directory on the server for files matching a particular pattern and download all such files from the server. Enabling this property makes the FTPGet adapter poll the Source directory using the scheduling configuration specified in Scheduler Configurations panel. The user has to make sure that the Source directory exists on server.

Icon

The properties Monitor Directory Configuration, Monitor Time Configuration and Use File not Found Exception in Monitoring are visible only when Monitor Directory property is enabled. 

Monitor Directory Configuration

Click the ellipsis button to configure Monitor Directory.


Figure 21: Monitor Directory Configuration properties

Source directory

The directory on the FTP server containing the files to be downloaded. FTPGet polls this directory using the scheduling settings configured in the Scheduler Configurations panel.

File name patterns

The type of files in the Source directory which are to be picked up and downloaded. This property accepts multiple file name patterns separated by pipes. Example: *.txt|*.xml|*.exe

Use additional directories for processing
Icon

If additional directories are required to be created automatically, then it's necessary to enable the Auto create temporary directories property in the Miscellaneous Settings section.

Enabling this property moves a file to Working directory when the file download starts, to Processed directory when the file download is successful, to Error directory when the file download has failed.

Icon

When the user does not possess the Move permissions, this property should not be enabled.

  • Working directory: This directory holds the files for which the file transfer is in progress.
  • Processed directory: This directory holds the files for which the download has been successful.
  • Error directory: This directory holds the files for which the download has failed.
Icon
  • FTPGet takes care of the creation of Working directory, Processed directory and Error directory on the server if the directories do not exist. Working, Processed and Error directories get created under the Current directory specified in Managed Connection Factory panel. If a user doesn't prefer moves and the creation of these extra directories then disable the property "Use additional directories for processing".
  • Monitoring is done by monitoring the source directory in the regular scheduling interval. The scheduler configurations can be defined by the user in Scheduler Configurations Panel and this is the only case that FTPGet component uses scheduler configurations.

Monitor Time Configuration

Click the ellipsis button to configure time-based filtering.

Time-based file filtering type

This property provides the capability of monitoring only specific files depending on their modification times. This property provides six options including 'NONE' based on which the files to be monitored could be filtered.


Figure 22: Time-based file filtering types

The behavior of each Filter Type is described in the following sections.

NONE

No filtering is applied to the files. Every file present in the Source directory is monitored.

TIME

Files whose last modification time is greater than the Base Time time is monitored. This ensures that only the files modified/added after the last scheduling cycle are monitored.


Figure 23: TIME Filter Type

  • Base Time: Base time in dd:MM:yyyy hh:mm format after which the changed files are to be downloaded.
  • Remote host time offset: If the FTP server and the component are not in the same time zone, the difference in the time zone of FTP Server time zone from the component's time zone should be specified in (+/-) hh:mm format.
HIGHEST MODIFICATION TIME

Files whose last modification time is greater than the highest last modification time found in the last scheduling cycle is monitored. This ensures that only files which are newer than the newer file already polled are selected.


Figure 24: HIGHEST MODIFICATION TIME Filter Type

Icon

Description of the attributes Base Time and Remote host time offset are the same as explained in TIME filtering type.

MINIMUM AGE

Files whose time difference between the current scheduling time and the last modified time greater than the minimum age is monitored. This ensures that the file modification time is at least Minimum Age earlier than the current time.


Figure 25: MINIMUM AGE Filter Type

  • Minimum age: The minimum age of the files which are to be monitored.
  • Remote host time offset: Refer description in TIME filtering type.
MAXIMUM AGE

Files whose time difference between the current scheduling time and the last modification time is less than the specified max age will be monitored. This ensures that the file modification time is not older than MaximumAge time for the current poll.


Figure 26: MAXIMUM AGE Filter Type

  • Maximum age: The maximum age of the files which are to be monitored.
  • Remote host time offset: Refer description in TIME filtering type.
LAST_RECENT_POLL_TIME

Files whose last modification time is greater than the last recent poll time will be monitored. This ensures that only files which are newer than the last recent poll time in the system are selected.


Figure 27: LAST RECENT POLL TIME Filter Type

Icon

Description of the attributes 'Base Time' and 'Remote host time offset' are the same as explained in TIME filtering type.

Example
Let us say if the source directory contains 4 files named a.txt, b.txt, c.txt, and d.txt. The scheduling interval is 3 minutes and the first poll is going to start at 11:00:00 (These scheduling settings can be configured in Scheduler Configuration Panel, please refer the section "Scheduler Configuration Panel" for more details)

Atfirstpollallfiles will be monitored irrespective of the value of the property "Time-based file filtering type".

Consider that the files have last modification time as below:

  • a.txt—11:00:16

  • b.txt—11:00:30

  • c.txt—11:02:10

  • d.txt—11:02:50

Following are the various outcomes for different filtering types. If the property Time-based file filtering type is set to:

  • TIME: All the files will be monitored in the next poll (which is going to poll at 11:03:00), since all files are modified after the last poll.
  • HIGHEST_MODIFICATION_TIME: All files will be monitored in the next poll (which is going to poll at 11:03:00), since all files are having the last modification time greater than the highest last modification time found on last poll (Component will keep the track of highest last modification time found in the poll).
  • MINIMUM_AGE:  If minimum age is set to 5 min, then no files will be monitored in the next poll (which is going to poll at 11:03:00). Files a.txt and b.txt will be monitored in the scheduling which will be going to poll at 11:06:00, and the files c.txt and d.txt will be monitored in the scheduling which will be going to poll at 11:09:00, because the files monitored in the particular scheduling have been modified at least 5 min ago from the scheduling time.
  • MAXIMUM_AGE: If maximum age is set to 2 minutes or 120000 milliseconds, then the files c.txt and d.txt only will be monitored at 11:03:00, because their last modified time is less than 2 minutes from the current system time.
  • LAST_RECENT_POLL_TIME: All the files will be monitored as all the files have modification time greater than the last recent poll time which is 11:00:00.
Use File not Found Exception in Monitoring

If this property is enabled, it throws a 'file not found' exception if no files matching the specified pattern and time filters are found in the monitored directory.

Target Settings

Following is the option available for Response Type - 'Data'.

Send XML output?


Figure 28: Interaction Configuration with response type - 'Data'

This property appears only when the response type is Data. When disabled, the component sends out only the downloaded content on the output port. When enabled, an XML message containing the downloaded content and download status details are sent out.

Refer to Figure 33 and Figure 34 for the sample input and output respectively when this property is enabled. Refer Figure 35 and Figure 36 which show the sample input and output when this property is disabled.

Following are the options available for Response Type - 'File'.


Figure 29: Interaction Configurations with response type - 'File'

Parent Directory on the local system

Path of the parent directory relative to which a relative path ofTargetdirectory would be computed.

Target Directory Configuration

Click the ellipsis button to configureTargetDirectory.


Figure 30: Target Directory Configuration properties

Target directory

Directory on the local system to which the file(s) is/are to be downloaded.

Icon

This property allows relative paths which would be computed relative to the directory specified for Parent Directory on the local system.

Auto create Target Directory

Enable this property to create target directory. If this is not set, the directory has to be created explicitly.

Use Temporary target Directory

If this property is enabled, then the FTPGet adapter will use a temporary target directory for intermediate processing. If you do not prefer to create extra directories in FTP server, disable this property.

Temporary target directory

This property is visible only when the property Use temporary target Directory is enabled. Directory on the local machine which the FTPGet component uses for intermediate processing during file downloads.

Icon

This directory should not be same as Target directory.

Action if same file exists

Action that must be taken if the target directory already has a file with a name that is same as the file that is being downloaded. The behavior will be dependent on the selection as shown below.


Figure 31: Append settings

OVERWRITE

The file being downloaded overwrites the one in the target directory.

RENAME

The name of the file that is being downloaded is changed based on the properties given below.


Figure 32: Append settings

  • Append date-time format postfix: When existing files in the Target directory are not to be overwritten, FTPGet provides the flexibility of downloading the content into a new file whose name is in the format <NameOfExistingFile_CurrentDateTime>. The format in which the date and time are to be appended should be specified as a value for this property.
    Example: If the date-time format is specified as MMddyyyyHHmmssss for the file Sample.txt, the target file created would be Sample_0305200811300013.txt.

  • Append counter: Enabling this property appends a counter along with the Date and Time to target file name when the target file is not to be overwritten.
    Example: A sample file name can be Sample_0305200811300013_0.txt.
APPEND

The file being downloaded gets appended to the one in the target directory.

Auto Create Target Directory

Enable this property to create target directory. If this is not set, the directory has to be created explicitly.

Miscellaneous Settings


Figure 33: Miscellaneous settings properties

Auto create temporary directories

The temporary directories created for processing of the component are created based on this property. If this property is not enabled then the temporary directories must be created explicitly.

Process Files based on Modification Time

This option is used to retrieve the files in a specific order.

Icon

This option is available only if you choose either of the following:

  • Allow File Patterns in Input Request option under Source Settings section.
  • Monitor Directory option under Monitoring Settings section.
File Processing Order

Options to process files based on Modification Time:

  • Oldest to Newest
  • Newest to Oldest

Testing the Interaction Configurations

The interaction configurations can be tested by clicking the Test button in the panel.
Example 1: Sample input and output when the Response type is set to 'Data' and Send XML Output? is enabled.


Figure 34: Sample input sent from CPS


Figure 35: Sample output

Example 2: Sample input and output when the Response type is set to 'Data' and Send XML Output is disabled.


Figure 36: Sample input sent from CPS


Figure 37: Sample output

The FTP server can be configured in the Connection Configuration property under Connection Properties sections in Managed Connection Factory panel of CPS.


Figure 38: Sample FTP server configuration

Server connection can be tested from within the CPS by clicking on Test in the connection properties panel.


Figure 39: Sample connection test result indicating success

Sender information can be configured using Target Directory Configuration property under Target Settings section in the Interaction Properties panel.


Figure 40: Sample Get information configuration

The configuration can be tested by getting a text file when you click on the Test option in the Interaction Properties panel.


Figure 41: Sample input


Figure 42: Sample response

Input and Output Schema

When the response type is 'File', the file to which the downloaded content is to be written should be specified in the input request to the adapter. The figure below illustrates the input and output schema structures when the response type is a File. Table 1 and Table 2 provide the descriptions for the schema elements of the input and output schema structures respectively.

When Response Type is 'File' the Target Settings properties appear as in Figure 27.

Icon

Input port appears only when Monitoring Property is disabled.


Figure 43: Input schema structure for the response type – File

Schema Element

Description

LocalDirectory

The directory on the local file system where the file needs to be written.

LocalFileName

The name of the file to which the downloaded content is to be written.

LocalPath

Path of the local file to which the downloaded content is to be written.

Icon

When LocalPath is specified, LocalDirectory and LocalFileName need not be specified.

RemoteFile

Name of the remote file on the FTP server which is to be downloaded (including path).

TransferType

Type of data transfer (ASCII or Binary)

Table 1: Input schema element descriptions for - File response type


Figure 44: Output schema structure for the response type – File

Schema Element

Description

LocalPath

Path of the local file to which the downloaded content is written

RemoteFile

File on the FTP server which has been downloaded

TransferType

TransferType mentioned in the input

BytesTransferred

The number of bytes transferred

TotalBytes

The total number of bytes transferred

ReplyCode

The reply code sent by the FTP server

ReplyText

The reply text sent by the FTP server

Table 2: Output schema element descriptions for - File response type

When Response Type is 'Data' the Target Settings properties appear as in Figure 26.

When the response type is chosen as 'Data', the Target settings in the Interaction Configurations panel change as shown in the below figure. Now, only one property Send XML output? appears in the Target Settings. When the property Send XML output? is enabled, the component sends out an XML message which comprises the downloaded data and Download status details. When disabled, the component sends out just the downloaded content.

Icon
  • Input port appears only when Monitor Directory is disabled.
  • Output port do not have any schema defined on it when the Send XML output? is disabled.

Below figures show the input and output schema structures when the response type is Data. Please refer Table 1 for the description of input schema elements and Table 2 for the description of output schema elementsexceptData.  When the response type is Data, this element holds all the downloaded content.


Figure 45: Input schema structure for the response type - Data


Figure 46: Output schema structure for the response type - Data

When the property "Use Connection details from input" is chosen, an additional element ConnectionFactorySettings is added to the input schema, as shown in the figure. Properties that are used to create the connection are present under this element.


Figure 47: Input schema with ConnectionFactorySettings

Icon

An input/output operation may get hung for an indefinite time waiting for a request to return due to various reasons like network fluctuations of the mounted directory. To prevent this indefinite waiting time, it is recommended to configure ConnectionTimeout in the runtime arguments. Refer to the Configuring Connection Timeout section in the Common Configurations section.

Functional Demonstration

Scenario 1

Receive files from a remote directory on the FTP server and save it in a local directory.
Configure the FTP Get as described in Configuration and Testing section and use feeder and display component to send sample input and check the response respectively. In the Interaction Configuration choose the option File.


Figure 48: Configuration the FTP Get

Sample Input


Figure 49: Demonstrating Scenario 1 with sample input

Sample Output


Figure 50: Demonstrating Scenario 1 with sample output

Scenario 2

Receive files from a remote directory on the FTP Server and send it out as data in the output message.
Configure the FTP Get as described in Configuration and Testing section and use feeder and display component to send sample input and check the response respectively. In the Interaction Configuration choose the option Data

Sample Input


Figure 51: Demonstrating scenario 2 with sample input

Sample Output


Figure 52: Demonstrating scenario 2 with sample output

Scenario 3 (Scenario 1 Using SFTP Protocol)

The following steps give a brief description of server settings and FTP protocol. Here, you provide steps to test FTPGet adapter using vsftpd server which is installed in Linux.
Steps to produce:

  1. Install vsftpd server by executing the following command on command prompt:

    yum install vsftpd
  2. Generate keys pairs (both RSA and DSA) and store in ~/.ssh/ directory. For the key generation, you can use the following commands:

    ssh-keygen -t rsa (for RSA type key generation) 
    ssh-keygen -t dsa (for DSA type key generation) 
    Icon

    While executing the above commands, file name and password for the file has to be provided. If the file name provided is id_rsa, then two files will be generated named id_rsa.pub and id_rsa which are public key and private key respectively

  3. Next, install the public keys on the server. Private key has to be with the client which is used to log in. The server will authenticate the private key using its public key. This type of client-authentication is called Public Key Authentication. The installation of the public key in the server can be done by the executing following command:

    ssh-copy-id -i ~/.ssh/id_rsa.pub root@localhost
    ~/.ssh/id_rsa.pub is rsa public key file path and root@localhost is the server (here we are using the same machine).
  4. Now, change the following in sshd_config file (this file can be located in etc/ssh/ folder). Set RSAAuthentication or DSAAuthentication based on the key file type used.
  5. Add the following line IdentityFile ~/.ssh/id_rsa in the ssh_config file which was located in /etc/ssh. (if already Identity File is set to some other file, then it has to be modified)
  6. Now, restart the servers using the following commands.

    /etc/rc.d/init.d/sshd restart /etc/rc.d/init.d/vsftpd restart 

    Now the server is ready to accept SFTP protocol to log in.

Configuring FTPGet Component

The configuration of the connection properties of FTPGet component for SFTP protocol is shown in the figure below, and the remaining procedures are same as explained in Scenario 1.


Figure 53: Connection Properties attributes

Use Case Scenario

In the retail television example, media production requests are received on an FTP server and are downloaded using the FTP Get component.


Figure 54: Demonstrating scenario

The event process demonstrating this scenario is bundled with the installer. The bundled process shows it as a File Reader component instead of an FTP Get component.
Documentation of the scenario and instructions to run the flow can be found in the Help tab of flow when open in Studio.

Scheduling

In the FTPGet component, scheduling cannot be directly enabled from the scheduling panel. To enable scheduling, in the Interaction Configuration panel, enable Monitor Directory property under the Monitoring Settings section and provide the supporting attributes that appear to do monitoring. Once these are configured, the scheduling panel details can be modified as required.

Useful Tips

  • In case the local file name is not specified in the input message, then the local file name is extracted from the remote file name. The directory shall be the one specified in the CPS for Target Directory property.
  • The remote file path is relative to the FTP server.
  • The component runs on the peer server and therefore the file paths and directories mentioned in the CPS should be valid on the machine where the peer server runs. If the component fails over to another peer, ensure that the machine on which the secondary peer server runs does have the same path available.
  • If an error with error code 550 occurs during request execution, the component logs the message as resource warning but not as an error. To get these messages as errors, the property "Throw fault on warnings" must be selected for request processing error in Error Configuration panel.
Icon

To understand the service better, refer the SFTP Monitoring example which demonstrates FTPGet service features.

Adaptavist ThemeBuilder EngineAtlassian Confluence