Fiorano Enterprise Repository

The Fiorano Enterprise Repository acts as an Enterprise-wide registry and repository for a Fiorano 10 Deployment. The repository is maintained by default in a File based data store. The FER (Fiorano Enterprise Repository) resides within the Fiorano Enterprise Server and manages the following:

• Event Process
• Pre-built Components and user defined components.
• Peer Configurations
• Principals and Access Control Lists
• Runtime State
• State Based Workflow Documents
• Events
• Alerts
• Policies

Event Process Repository

The Event Process Repository manages all the pre-built Event Processes and composed Event Processes. This is a layered repository over a System Store and User Store. The System Store stores the default set of Event Processes which are shipped along with the product and the User Store stores all the Event Processes which are subsequently composed by users. The location of the System and User Event Process repository are:

• System Repository: %FIORANO_INSTALL_DIR%\esb\server\repository\applications
• User Repository: %FIORANO_HOME%\runtimedata\repository\applications

The system repository is read-only and the entire user defined Event Processes are stored in the user repository. If you wish to move user-created Event Processes from one installation to another compatible installation, the application folder can be copied from one location to another. If format-compatibility does not exist between the versions, then the User has to export all the User Defined Applications in order to import them into the new installation. The Application Repository can be copied and backed up at anytime.

The Structure of the File Based Store for an Event Process is explained briefly below. An Event Process is arranged into Files and Directories.

• EventProcess.xml – Main index file which stores the details of the components. This file is linked to other files that contain configurations and schemas used for communication as well as for the layout of the workflow diagram.
• Config – Stores all the component configurations used in the Event Process.
• Schemas – Contains the schemas referred by the components used to specify the format of the expected input / output messages.
• Env – Deployment and environment properties.

An Event Process must be treated as a single unit/entity and has to be copied as such. Copying partial data from folder will corrupt the Event Process data.

Component Repository

Components or Services in Fiorano 10 are business functions, each with a set of configurations, which execute a single unit of work. Components can be developed and registered with the Fiorano Component Repository. A Registered component can be re-used any number of times in an Orchestrated event process. Supports for versioning of components are built into the repository.

The Component repository manages all pre-built components and the custom-created components. This repository is a layered over the system repository and the user repository. The System repository is a read-only repository that stores all the prebuilt components, while the user repository stores all user-defined custom components. The locations of the two repositories are:

• System Repository:   %FIORANO_INSTALL_DIR%\esb\server\repository\components
• User Repository: %FIORANO_HOME%\runtimedata\repository\components

A Component repository can be backed up at runtime and can be version controlled. This repository can be retrieved and replaced at any time, but the changes will not be affected until:

1. The application which uses any of the changed/replaced components is stopped.
2. The componentcache property found in the properties panel of the event process is disabled and
3. A Connectivity and Resource Check (CRC) is done for the Event Process.

Each peer server has its own local cached copy of the component and the above steps are necessary for the Peer Server to update the component from the Enterprise Repository.

Migrating the component repository between compatible Fiorano versions involves copying the components directory to the corresponding location in the new installation. The System repository should not be copied as it is read-only and there can be a newer version of prebuilt components in the new installation. Please refer to the Installer Release Notes for the component compatibility matrix between different versions. For migrating custom components between non-compatible versions, each custom component has to be explicitly exported and imported into the new installation.

The Component repository is organized in files and libraries. Every component is represented by a directory with its GUID as the directory name and each directory has a sub-directory within it which stores the version of the component. Every component has the libraries required for that particular component, together with a Service Descriptor. A Service Descriptor describes the structure of the component and its input ports and output ports.

Peer Repository

The Fiorano Enterprise repository also manages the configurations of all Peer Servers on the Fiorano network. Each Enterprise Server has its own Peer repository where it tracks the configurations of all Peers connected to it. When a Peer Server connects with the Fiorano Enterprise Server, the configurations are referred to in the Peer repository of the Enterprise Server to which the Peer is connected and the Peer fetches the configuration from the corresponding repository. If the Peer registers with the Enterprise server for the first time, the Peer configuration is pushed into the Peer Repository of that particular Enterprise Server.

Peer HA Profiles have a special property which inhibits this behavior of fetching the configuration from the Enterprise repository. Since two different HA Profiles connect with the same name, the configurations cannot be fetched and loaded in this case. This special property is internal to Fiorano and is not exposed to the user. When one of the default HA Profiles is copied to create a new profile, this special property is carry forwarded. This property cannot and should not be set in a standalone profile.

The Peer repository is a transient repository built dynamically as the peers get registered, and hence any migration effort should not involve copying this repository. Additionally, this repository should not be backed up and restored. It is recommended that whenever a Peer profile is changed, the profile should be cleared form this repository for the change to take effect otherwise the configuration present in the Peer repository gets pushed to the Peer Server and the Peer Server loses the changes made to its local profile.

The peer repository can be located at: %FIORANO_HOME%\runtimedata\repository\EnterpriseServers\{FES_profile_name}\FES\Peers\.

Each peer profile is a directory with the name of the Peer and with a set of definition and configuration files inside its folder.

Security Store

The Fiorano Enterprise repository manages the Users, Groups and Access Control lists for each principal. This is a File Based Store and the format is internal to Fiorano and it is not visible to the end user. The Security Store can be found at: %FIORANO_HOME%\runtimedata\EnterpriseServers\{FES_profile_name}\FES\run\SDB. Fiorano has a centralized user and permissions group store, any changes to this user and permission group store are propagated to all connected Peer Servers, whose local security stores are then updated. This store contains information about the users, groups and the access control list for each permission.

The security store can be backed up at runtime; however, restoring the store requires the Enterprise to be stopped. The security store can be migrated between compatible versions of Fiorano. For non-compatible versions of Fiorano; all of the users, groups and access control lists need to be recreated.

Runtime Store

The Fiorano Enterprise Server stores the current state of an Event Process and other Metadata such as the system level destinations created within the admin runtime store. When an Event Process is started, the state of the Event Process is stored in this admin store and is restored whenever the Fiorano Enterprise Server is started. State of the Event Process can either be RUNNING or NOT_RUNNING and this State needs to be persisted in order to restore the state of Event Processes on Server restarts. When servers are restarted, the Enterprise Server refers this repository and automatically starts those Event Processes that were running the last time the servers were able to persist in the state of the Event Process. There is no guarantee that the state will be the very same state as when the servers were brought down or crashed since it may have persisted in the state in which it may have crashed. The Fiorano Enterprise Server automatically caches the state of the Event Process when there is a change to it.

The term state here refers to key information relating to the deployment of an Event Process and not the actual runtime data flowing between components participating the Event Process. The local naming and directory module binds the objects to a name in this runtime store. This is a single file located at:

%FIORANO_HOME%\runtimedata\EnterpriseServers\{FES_profile_name}\FES\run\admin.dat

If the state of all Event Processes needs to be cleared, then the admin.dat can be deleted. This is a transient data structure which should not be backed up and restored. This data is stored in an internal format and is not visible to the end user.

If the Enterprise Server is running in High Availability mode, its state together with that of it’s backup/secondary server, is maintained in:

%FIORANO_HOME%\runtimedata\EnterpriseServers\{HA_PROFILE_NAME}\{PROFILE_TYPE} \FES\run\haStatus.dat

This is a transient data structure used to maintain the current state of an HA server as well as it’s previous state. The file format is internal and is not readable; as such, this data file should not be backed up or migrated to a different installation. This data store is common to all servers running in High Availability mode. All Peer Servers running in HA mode have a similar data store with identical semantics.

The runtime data store that stores the pending messages and the destinations which are responsible for the actual Peer-to-Peer message traffic between different components are present in each Peer Server in the following location:

%FIORANO_HOME%\runtimedata\PeerServers\{PROFILE_NAME}\FPS\run

This transient message store can be backed up and migrated if there are no changes in the Event Processes or the components. The pending messages will be processed the next time the servers are up after migration. 

State Based Workflow Repository

The State Based Workflow (SWB) or Document tracking creates Document events that are raised in the Peer Servers and forwarded to the Enterprise Server. The Enterprise Server is responsible for storing the Documents for auditing and tracking purposes.

SBW documents are pushed into a temporary persistent cache from the Peer Servers, from where they are picked up by the Enterprise Server and inserted into the configured SBW Database. By default, this database is configured as the internal H2 database and stores the record in its proprietary format in the file system.

Persistent cache:

Persistent cache is a transient data store into which tracked documents are written temporarily before they are moved to the configured external database for storage. This cache can be found at:

%FIORANO_HOME%\runtimedata\EnterpriseServers\{FES_Profile_Name}\FES\run\db\PersistantDB\PQ.Tifosi_Enterprise_Server_SBW_Events

This cache should not be cleared manually; the size of this cache can build up when the configured Document tracking store is not available. As such, the SBW configured database cannot be down for an extended period of time. If it is down for an extended period of time, the document tracking configurations have to be changed to point to a different external database or else the cleardb script will be used to clear the persistent cache of the SBW Documents. The Persistent cache is a transient data store and the data is stored in an internal record format which is not visible to the end user and should not be migrated.

Document Tracking H2 Database
%FIORANO_HOME%\runtimedata\EnterpriseServers\{FES_Profile_Name}\FES\doctracking_db

This is the default SBW data store shipped with the Fiorano Platform. The database is an H2 data store, which should not typically be used in a production setting. In Production, an external database should be configured to store and retrieve the tracked documents and periodical cleaning of the tracked documents should be scheduled and executed with the help of Fiorano scripts. The database can be archived for auditing purposes and can be backed up regularly.

For more information on the structure of SBW database and for instructions on how to configure a different external database to be used for SBW tracking, please refer to section Document Tracking.

Disabling SBW Tracking

To disable SBW Tracking globally, perform the following steps:

1. Login to FES-JMX and locate Fiorano > Esb > Sbw > SBWManager > SBWManager > config.
2. Set the property named 'EnableDocTracking' to 'no'.

Event Repository

The Fiorano Platform raises Events for all meaningful operations that occur in the system. These System Events are collected on the Enterprise Server and the state of each artifact (Event processes, components, server status etc) is determined by the sequence of such events. All captured System Events are inserted into a configured database. If the component is configured to raise the monitoring events on the various performance metrics, then these events are also stored in the Event repository. The default database for the Event repository is the H2 database shipped by default along with the Fiorano Platform.

Custom Event Listener

Custom applications can be written to listen to the System Events and SBW Events. For more information, see section Events Tracking.

Refer to RTL Java documentation of com.fiorano.esb.fes.rtl.events.FioranoEventsManager for more details on registering with the Enterprise Server for listening to System and User events.

Alert Repository

The Fiorano Enterprise Repository manages all Alert configurations. The Alert configurations are looked up when a notification is being executed. This is a File Based Repository located at:

%FIORANO_HOME%\runtimedata\repository\Notifications

Each alert is represented by a single XML file listed in the above directory. The name of the file is the unique identifier of the Alert. The structure of the document varies depending upon the transport over which the notification is configured. Fiorano does not recommend changing values directly in the XML or any interaction with the OS in a way that affects the file system. In particular, an “Antivirus” or any other program that locks files or hinders the creation and deletion of files should not be run over this repository (or any of the Fiorano repositories).

The Alert repository can be backed up and versioned in any source control and restored at anytime during the Fiorano Server Execution. The Updated notification from the Repository will be picked up automatically at the next execution of the notification. This repository can also be migrated from one version to another through copying the file system provided the repositories are claimed interoperable in the release notes of newer versions of Fiorano. Please check the release notes for the interoperability matrix on this repository. 

The Alerts are queued up as jobs which are then picked up by the Job Manager and executed serially. There is no guarantee that the alert will be executed within a specific period of time. If the notification job crashes in the middle of the execution the data will be lost and will not be re-executed when the server starts up. However, the pending list of notifications will continue to be executed.

Policy Repository

Fiorano governance solutions use a Policy-Based approach. Policies can be created for a condition and on achieving the condition a notification over a transport is executed. The Fiorano Enterprise Server manages the policies in a default File-Based repository. This Policy Repository is internally dependent on the Alert Repository for fetching the notification information which it then executes.

%FIORANO_HOME%\runtimedata\repository\Policies

Backlog policies are one type of policy that register for a particular count of the pending requests at an execution port of a service and execute notifications based on the count. The backlog policies are stored in the following location:

%FIORANO_HOME%\runtimedata\repository\Policies\Backlog

For each type of policy, a directory is created inside the Policy repository. Each backlog policy is stored in an XML file whose name matches the name of the policy ID. Each type of policy has a specific XML structure that includes the condition details and the alert lists to be executed when the condition is met.

The Policy repository can be backed up and versioned in any source control and can be restored at any time when the Fiorano server is not running. The Updated policies from the Repository will be picked up automatically from the web console upon a refresh. This repository can also be migrated from one version to another through copying the file system provided the policy structures are claimed interoperable in the release notes of the newer version of Fiorano. Please check the release notes for the interoperability matrix on this repository. 

A Policy can be either in an ACTIVE or an INACTIVE state. The current state of the policy persists along with the policy. Once a policy is applied, a restart of the servers automatically re-applies the policy and the Peer Servers get notified about these policies. Backing up and restoring the policy repository will also restore the state of the policy at the time it was backed up.

Changes in Repository Location

A Layered repository approach is implemented after version 'Fiorano 2007 SP1'. This has brought the following changes in the location of the repository: