FioranoMQ JMS Server

Architecture


Introduction

Today's enterprise networks typically deploy hundreds of applications from different vendors. There is little standardization of communication protocols between individual distributed systems and exchanging information between applications from different vendors is surprisingly difficult. The lack of a standard platform for network-centric computing increases the cost and complexity of developing and deploying distributed systems. Moreover, the explosion of Internet and rapid proliferation of computer networks have created the need for a comprehensive software platform that specifically addresses the requirements of event-based network-centric applications.

Java Message Service (JMS) provides a common technique for Java programs to create, send, receive and read messages in an enterprise messaging system. The FioranoMQ JMS Server is an enterprise messaging platform which is designed over the Java Message Service Standard.

Java Message Service is a set of interfaces and associated semantics that define how a JMS client accesses the facilities of an enterprise messaging product. A Java Message Service provider is an entity that implements the JMS specifications and the APIs for providing an enterprise messaging solution. The FioranoMQ JMS Server is such a Java Message Service provider that has support for all the JMS specifications. In addition, it provides numerous proprietary features such as scalable connection management, file based data store and load balancing that make it one of the fastest, most scalable JMS servers currently available.

The Java Message Service messaging model consists of two basic domains PTP (point-to-point) and Pubsub (publish/subscribe).

Point-to-point (PTP) products are built around the concept of message queues. Each message is addressed to a specific queue; clients extract messages from the queue(s) established to hold their messages. Publish and subscribe (Pub/Sub) clients address messages to some node in a content hierarchy. Publishers and subscribers are generally anonymous and may dynamically publish or subscribe to the content hierarchy. The system takes care of distributing the messages arriving from the multiple publishers of a node to its multiple subscribers.

The FioranoMQ JMS Server implements both the messaging domains as well as the unified domain concept.

FioranoMQ JMS Server


System Architecture
Large scale Internet and Intranet applications require hundreds if not thousands of concurrent connections to the server, a number that is expected to grow even higher over the next few years. As such, an enterprise level messaging platform needs to be able to scale smoothly to handle the increasing numbers of client connections.

Hub and Spoke Topology
The FioranoMQ JMS Server's design departs from the traditional point-to-point communications architecture to follow a "hub-and-spoke" design. The FioranoMQ JMS Server acts as the hub and JMS enabled applications as the spokes. The hub-and-spoke topology has several benefits:

Passing all the messages from the applications through the server allows the advantages of centralized administration, security, and routing of messages. For the system administrator, this implies all communications between the applications can be monitored and maintained from a single location. For the developers, this implies each application only needs to interface with the Java Message Service compliant FioranoMQ JMS Server (which loosely couples applications).
A comprehensive security model becomes much simpler to arrange, when all communications go through a single point.
Message routing prevents users from having to screen all the messages locally to identify those that they need. Additionally, an application that publishes messages does not need to know the address of each subscriber.

Figure 1 illustrates the advantages of a hub-and-spoke topology in terms of logical connections between applications.

Comparing point-to-point with hub-and-spoke topology

Figure 1: Comparing point-to-point with hub-and-spoke topology

In Figure 1, with six applications communicating, the system requires 15 connections with the traditional point-to-point design and only six connections with the hub-and-spoke topology. The point-to-point model requires each application to include the code to interface with the five other applications; furthermore, each application requires modification if another application joins the network. In the hub-and-spoke model, each model only needs to include a single set of code for communicating with the FioranoMQ JMS Server at the hub. The hub decouples applications, thereby reducing system complexity and increasing system flexibility and adaptability.

Internal Architecture Details
The FioranoMQ JMS Server is a messaging server that implements both the publish/subscribe (Pub/Sub) as well as the point-to-point (PTP) subsets of the JMS API. Both of these subsets of the JMS API require some form of offline database to store persistent messages, which have been received but not yet delivered to all the client applications.

The system architecture of the FioranoMQ JMS Server is illustrated in Figure 2.

FioranoMQ Publish/Subscribe Server System Architecture

Figure 2: FioranoMQ Publish/Subscribe Server System Architecture

Following important points can be noted from the Figure 2 :

The offline database is implemented using the default file system of the platform on which FioranoMQ is installed. The database contains multiple sub-directories, one for each topic for which messages need to be stored. As messages are consumed (i.e. when they have been delivered to all registered subscribers), the size of the database is automatically pruned using an efficient purging algorithm.
If a publisher marks a message as persistent, then the message is always stored in the offline database. This protects the clients against JMS provider failures. In order to ensure that a persistent message is either delivered to the JMS provider for sure or not delivered at all, the sending thread in the client program waits until the message has been completely written into the offline database of the FioranoMQ server. Hence, if either the client or the server crashes, it is ensured whether the message has been written into the offline cache.
Messages remain in the offline cache until they have been delivered to all the registered durable subscribers. Information about all the durable subscriptions created on a given topic are stored in the offline cache. When subscriptions are removed (using the JMS "unsubscribe" call), the corresponding references are removed from the offline database.

The offline database storage subsystem ensures that all the persistent messages are delivered to all registered durable subscribers, with guaranteed onetime delivery. In case a provider failure occurs during message delivery or if the client application crashes, a persistent message is always redelivered. The redelivered message is marked, as required by Java Message Service semantics.

Note that Java Message Service allows non persistent messages to be lost in case some subscribers are not connected to the system. Therefore, if you are developing a distributed system and want to ensure that none of your messages are ever lost, it is best for you to:

Make all your subscribers Durable. Messages meant for a durable Subscriber are stored in the persistent cache even when the Subscriber is inactive.

Ensure that your client applications publish only persistent messages.

Thread Management in FioranoMQ JMS Server

In addition to the 100% pure Java implementation of the default connection manager, FioranoMQ JMS Server software incorporates a special Scalable Connection Management (SCM) module. This allows a single FioranoMQ JMS Server to support several thousand concurrent client connections on a variety of operating platforms, such as Windows NT, Solaris and other UNIX platforms. The FioranoMQ JMS Server allows the user to choose between the default Java connection management and Scalable Connection Management.

Using the default Java Connection Manager, each server instance runs in its own JVM. Clients communicate with the server through sockets, with each client connection using two threads.

A single FioranoMQ JMS Server can handle as many concurrent connections as are supported by the JVM, under which the server runs. It is the responsibility of the administrator or the application designer to ensure that the clients connect to servers in the cluster in such a manner that the client load is automatically distributed between the servers.

As more clients connect to the server, the number of threads used on the server goes up linearly. This is because client connections are persistent and are not aborted / removed even if the client is not sending or receiving any messages. In case the server JVM runs out of threads, an error is reported and the FioranoMQ JMS Server refuses the requested client connection. This condition implies that the thread limit for the JVM on the server machine has been reached. This number depends on the JVM being used, and varies depending on the target platform. On Solaris, our tests (April 2000) show that over 3500 concurrent connections can be supported using JDK 1.2, while Windows NT environments are a little less scalable. As JVM technology improves, the total number of concurrent connections that can be supported on a single server will continue to rise. The FioranoMQ JMS Server is only limited by the JVM that it runs under, and not by any internal server limits. If your application needs more concurrent client connections than are supported by a single server, you should use the Scalable Connection Manager and/or cluster multiple FioranoMQ JMS Servers connected together.

The thread limit is typically very dynamic and depends critically on the load on the server machine. By aborting compute and thread intensive applications, one can increase the number of threads available to the JVM, running the FioranoMQ JMS Server. Thus, concurrent client connections to a single FioranoMQ JMS Server are limited in number. The exact limit critically depends on the hardware configuration and the system software (for example, the JVM) running on the FioranoMQ JMS Server machine.

Memory Management and Publisher Throttling in the FioranoMQ JMS Server

Each client connection to the FioranoMQ JMS Server uses some amount of available memory. The FioranoMQ JMS Server implements internal memory management algorithms in order to keep track of the approximate amount of memory being consumed by the system. When the memory limit of the JVM is reached, the FioranoMQ JMS Server automatically starts throttling the publisher application(s) connected to the server and, where allowed by Java Message Service semantics, overflowing the internal buffers. The publisher throttling algorithms automatically governs the speed of running applications, ensuring that the server never gets flooded at any point and that total server throughput remains constant. Publisher throttling ensures that the server scales well as more clients are added to the server.

Typical FioranoMQ JMS Server Setup

In a typical case, the FioranoMQ JMS Server is used in standalone mode. During the setup process, the system administrator uses external tools and special administrative APIs to create a set of destinations on a FioranoMQ JMS server. As defined by Java Message Service, a Destination is either a Topic for Publish/Subscribe interactions or a Queue for point-to-point interactions. Each standalone FioranoMQ JMS Server owns a set of available destinations and clients connect directly to the server, as illustrated in Figure 3.

FioranoMQ Server <- JMS Client Applications

Figure3: Typical setup of FioranoMQ JMS Server

In the typical setup illustrated in Figure 3, all the clients connect to a single server. This setup, while adequate for a large number of applications, suffers from the inherent problem that multiple concurrent client connections can overload the server, resulting in a general slowdown in response time for all applications. To solve this problem, multiple FioranoMQ JMS Servers can be connected together, allowing clients connected to one server to exchange information with clients connected to another server.