Managing JVM Settings and In-memory Execution

Java Virtual Machine (JVM) is required not only to execute code but also to enhance speed by managing memory. JVM helps in improving the performance of servers and to eliminate out of memory error by managing memory.

JVM 32 bit Vs JVM 64 bit

While choosing a JVM, either 32-bit or 64-bit can be considered depending on the server machine architecture. A 32-bit Java Development Kit (JDK) can be used with 64 bit architecture but 64 bit JVM is highly recommended in production environment where the servers will need more memory. 64-bit JVM can be used when there is a huge memory requirement for a process and the amount of memory taken by JVM is generally more.
Ideally, servers can be run using 64 bit JDK and components can be set to use 32 bit JDK, and the memory being used can be monitored in Fiorano Dashboard.

Example: For a process that takes 1 GB memory in 32 bit where it is the maximum needed, prefer 32 bit, whereas for a process that needs around 3 to 4 GB, a 64 bit JVM is required because a 32 bit JVM can accommodate around 2 GB only.

Setting up Java

The Fiorano Platform needs Java Runtime Environment (JRE) to function properly. To setup this environment, refer the Setting up Java section.

Understanding Memory Usage

The JVM memory mainly consists of the segments: Heap and Non-Heap.

Heap Memory

Heap memory is the runtime data area from which memory for all class instances and arrays are allocated. Most of the data related to messages are stored in heap.
The heap size may be configured with the following VM options:

• -Xmx<size> - to set the maximum Java heap size. Set the maximum Java heap size so that there is 70% occupancy at the highest point
• -Xms<size> - to set the initial Java heap size. Set the minimum heap size so that the Java heap is 40% occupied at its lowest memory usage. By default, the maximum heap size is 64 Mb.

Non-Heap Memory

Used by Java to store loaded classes and other meta-data (Java class definitions etc) corresponding to the messages. It stores per-class structures such as runtime constant pool, field and method data, and the code for methods and constructors, as well as interned Strings.
The Non-heap size may be configured with '-XX:MaxPermSize' option.

Fiorano Dashboard

In Fiorano Dashboard, the split-up of Heap and Non-Heap Memory usage for each Server Type and CCP enabled components are available.

Server Example:

Figure 1: Screenshot of Server Status in Dashboard

Component Example:

Figure 2: Screenshot of microservice Details in Applications taken from the Dashboard

In the screenshot of Server Status (Figure 1) taken from the Dashboard, the Memory Usage of fes server reads as below:

• Heap: 137MB/910MB
• Non-Heap: 33MB/304MB

Below are the implications of the above values:

• 137MB/910MB of Heap implies that 123MB of memory is presently being used and 910MB is the maximum amount of memory that can be allocated.
• Total memory being used by fes is the sum of Heap and Non-Heap used memory, that is, 137MB + 33MB =170MB.
• Total memory that can be allocated is, similar to the used memory, the sum Heap and Non-Heap allocated memory: 910MB + 304MB = 1214MB.

Heap size allocated for maximum memory can be customized, as per requirement, by configuring the Heap size, which is explained in the next section.

Configuring Memory

Heap size allocated for maximum memory can be customized, as per requirement, by configuring the Heap size.

Server Configuration

The allocation of memory for the JVM is specified using -X options when starting the server. These options can be configured in server.conf file in %FIORANO_HOME%/esb/server/bin/. Open server.conf file and change the values for –Xms and –Xmx argument under <jvm.arg> tag. Save the file and restart the server.

Component Configuration

Memory used by a component can be restricted or expanded based on the requirement. This can be done by appending appropriate "–Xms" and "-Xmx" values with JVM_PARAMS value present in Runtime Arguments tab under Properties.

Figure 3: Example: -DLOG_TOSTREAM=true –Xms40m –Xmx128m

Reducing the maximum allocated memory helps in deploying more components, but keeping the value too low can cause out of memory errors.

Using same JVM settings in Multiple Microservice Instances

After changing the JVM_PARAMS value, as described in the previous section, clicking No in Update all Service Instances dialog box applies the setting only to the presently selected Microservice Instance(s). Clicking Yes provides Select Service Instance window where the same setting can be applied to multiple Microservice Instances from all the Event Processes.

Figure 4: Updating Microservice Instances

Using same JVM settings in Multiple Environments

To use a particular JVM_Params setting across environments, save the configuration as 'named configuration' bu performing the following actions:

1. In Configuration Repository window, right-click Runtime Arguments folder and click Add Configuration.
2. In the Add New Configuration dialog box, choose the preferred environment name from the Environment drop-down.
3. Provide a name for the configuration in the Name field and click Next.
   
   
   Figure 5: Configuring Named Configuration
   
   
4. Click Finish to save the named configuration for the chosen environment
   
   
   Figure 6: Saving Named Configuration

Configuring Execution Type

This section provides guidelines on Managing System Resources across Peer Servers together with recommendations on in-memory execution. The number of components running on a Peer Server must be optimized to optimize the load on that Peer Server and to utilize machine resources effectively. Launch Options - Separate Process and In-memory are discussed below.

Using 'Separate Process' Launch Mode

The number of components that can be launched on a Peer Server depends on the RAM availability. Users can launch more components on a machine with greater RAM than on one with less RAM.

However, there is a practical limitation on the number of components that can be safely launched on a Peer Server since all the components are launched as child processes of the Peer Server process. As the number of components on a particular Peer Server increases, it becomes difficult for the Peer Server to launch and stop the components. This can result in Request Timeout exceptions.

To avoid these timeout exceptions there is a need to distribute components across different Peer Servers.
The default timeout for application launch/stop in the Fiorano Studio is 100 seconds. Any server call is timed out after this duration. This value is configurable and can be increased to avoid frequent timeouts.

Fiorano recommends launching 60-75 components per Peer Server on a machine with 8 GB of RAM. The limit can be slightly greater on a machine with more RAM.

'Separate Process' mode is preferred for components with high memory footprint such as DB, WSStub, and WSConsumer with 32-bit JDK. If the message payload (request size at runtime) is huge and transformation is complex, use XSLT components in Separate Process mode.

Using 'In Memory' Launch Mode

In case of 'In memory' Launch, components are launched with the JVM of the Peer Server of the local machine. The default Heap memory allocated for servers in the Fiorano environment is 512 MB. Server heap memory can be increased by changing the –Xmx parameter value by editing the server.conf file located at FIORANO_HOME\esb\server\bin. For 'In Memory' Launch, it is recommended that the server Heap memory should be increased to the maximum limit, though this has the effect of allowing less threads within the overall Peer Server process.

In case of 32-bit machines, a java process can span up to a maximum of 1.6 GB of memory. Assuming that the heap-size is set to the maximum, that is, 1.6 GB limit, the number of 'In Memory' components can be increased until the Peer server's memory usage reaches this point (or within safe limits of this point, say approximately 70% of 1.6 GB).

In case of 64-bit machines, the java heap memory is relatively high (approximately 4 GB) and a larger number of components can be launched in In-memory. However, one needs to be careful in assigning more memory to the Peer Server process since more Heap memory has an impact on the number of threads that can be created by the JVM. As the heap memory of a process increase, the number of threads that can be created is less; this can have an adverse effect on Peer Server performance. As such, the precise heap-memory allocated to the Peer Server JVM has to be determined by testing that particular configuration being run (that is, by running the full application to ensure there are no adverse effects on the system).

In-memory mode is preferred for components with less memory footprint such as XSLT and CBR. Server JVM will be used while using components in this mode.

Launch Mode can be changed to 'In Memory' from Execution Type drop-down present in the Execution tab under Properties.

Figure 7: In Memory Execution Type

Changing the Default Launch Mode

When it is clear that certain microservices in a particular environment are good to be in a different Launch Mode, these components can be made to stick to the preferred launch mode by changing the default value by editing their microservice in the Execution tab.

To change the default Launch Mode of a particular component:

1. In the Micro Service Repository panel, right-click the microservice for which default Launch Mode has to be changed and click Edit.
   
   
   Figure 8: Navigating to ServiceDescriptor.xml page to change default Launch mode

   Icon

   If the Micro Service Repository panel is not found in the eStudio screen, open it from Window > Show View.

2. Click Execution tab present at the bottom of the ServiceDescriptor.xml page.

3. Scroll right and select the required Launch Mode from the Preferred drop-down present under Launch Configuration section.
4. To apply the changes made, click Save (else click File > Save or press CTRL+S).
   
   
   Figure 9: changing the default Launch mode

Route Transformations

When user transforms XSLT component to route transformation, the transformation happens in peer JVM.

All the memory details can be checked in Dashboard and should be fine-tuned based on the requirement.