Just because Tomcat is a lightweight container with a small footprint doesn't mean it isn't ready to deliver big performance under real-world loads. Tomcat's built-in support for clustering, load balancing, and session persistence means that you can add more power to your network as you need it, allowing your site to scale with your user base.
In this article, we'll go over the basic concepts behind clustered architecture, such as load balancing and session persistance, look at some different ways to approach these problems, and then show you how easy it is to set up your own Apache Tomcat clusters.
How Clustering Works
Although clustering is most frequently talked about in relation to scalability, most modern clustering solutions actually attack a number of related issues in addition to simply providing more CPUs to serve requests. A typical clustering solution aims to provide not only scalability, but also high availability and load balancing. Before we move on, let's briefly clarify each of these terms:
Scalability and clustering are not the same thing. Rather, clustering is a method of achieving scalability. Scalability has to do with the ability of a server to efficiently process multiple concurrent requests simultaneously, with the stated goal that the time it takes to process an ever increasing number of simultaneous requests should be as close to the time it took to process the initial request as possible.
Clustering aims to provide scalability by spreading work out over a greater number of workers. Other methods of improving scalability are improving the hardware of the machines processing the requests, streamlining the data, caching frequent operations, and more.
Load balancing is a group of technologies aimed at distributing request load across a group of servers. Load balancing is a key component of a clustering solution, as it provides several services required to achieve the other goals of clustering.
To enable scalability, a load balancing implementation attempts to route requests to the server with the least amount of current load, for faster processing. To enable high-availability, which we will define next, a load balancing implementation must keep track of the status of its various servers, so that requests are never dropped.
Many load balancing solutions also take advantage of the fact that a server is now fronting the actual request processing software to provide an additional layer of security, ignoring and dropping malicious traffic before it can even reach the application servers.
Finally, the load balancing implementation makes the whole clustering structure functional by encapsulating the cluster within a virtual container, with one point of access. This means that the client attempting to access the web application served by the cluster never needs to know whether or not a cluster is being used.
High availability is a group of interrelated technologies and strategies with the aim of increasing the amount of time that the network is available to process requests. The most common of these techniques are failover, state replication, and load balancing.
Load balancing relates to high availability in two ways. First, it is a preventive measure; if a server never becomes overloaded, it will not fail. Secondly, it provides support to the failover mechanism by routing requests away from failed servers.
Failover refers to the ability of any server in the cluster to take over the work of any other server in the event that any server fails. Failover implementations can be divided into two categories - request-level and session-level.
Request-level failover is provided by the load balancer, and refers to the redirection of all subsequent requests after a server fails to a different server, to avoid any break in service.
Session-level failover, a more complicated proposition, involves replicating user sessions in such a way that in the event that a server goes down, another server or servers can take over the session information it was using, meaning that the user will not perceive any break in service. This is provided by using some form of session replication as a part of cluster.
One way to think about the structure of the cluster is as a sandwich - the load balancer is in front, distributing requests, the Tomcat servers are in the middle, serving requests, and the session replicator is in back, preserving the state data, in case anything goes wrong.
Setting up Clustering in Apache Tomcat
Let's get an example Tomcat cluster up and running. As Tomcat supports a few different options for load balancing and session persistance, we'll deal with these parts in separate sections, so we can dig a little deeper into the pros and cons of different set-ups.
Then, we'll put it all together, and walk you through setting up a simple Tomcat cluster with load balancing and session persistence from start to finish.
Load Balancing Options
You can approach the problem of load balancing in a variety of ways, and Tomcat supports most of them with a little wrangling. The two most common ways to balance load across a group of servers are to use a dedicated load balancing appliance, or a software/server solution.
Load Balancing With An Appliance
Load balancing network appliances are dedicated pieces of hardware that front a cluster and provide integrated load balancing capabilities. These devices use processors specifically designed for distributing high volumes of load efficiently, and also usually include a variety of compression, cacheing, and queueing options, as well as traffic shaping and security capabilities.
High end failover appliances can be very expensive, and can still represent a single point of failure for the network if scale continues to increase. Avoiding this problem brings the idea of HA load balancing into the equation, which uses such methods as hot spares to provide back-up.
While some users choose to utilize hardware load balancers with Tomcat clusters, other solutions are more common. For this reason, as well as the fact that each appliance vendor has slightly different set-up procedures for their devices, this article does not cover the configuration process for this approach. However, your vendor should be able to provide you with the proper documentation and support.
Load Balancing With a Server/Software Combination
The concept of the server/software approach to load balancing is basically the same as that of the appliance, but rather than using a dedicated device, it relies on a dedicated server or group of servers running one of a number of proxy solutions with load balancing capabilities.
Apache HTTPD and mod_jk/mod_proxy
The most popular server/software set-up for Tomcat clustering is to front a cluster of Tomcat servers with an Apache Web Server running either the mod_JK or mod_proxy connector module. These modules, which are also often used simply to provide basic interoperability between Apache Web Server and Tomcat, also each include built-in load balancing capabilities.
At one time, it was common practice to favor mod_jk over mod_proxy; this was because mod_jk was developed as part of the JK project, a Tomcat subproject aimed at improving connectivity between Tomcat and various web servers, and had support for AJP, an efficient protocol developed specifically for meta-data-rich communication between Apache Web Server and other types of servers.
The speed of AJP made this protocol preferable, and was a big vote on the side of mod_jk. However, when mod_proxy was refactored in Apache Web Server 2.2, it was vastly improved, and included new sub-modules offering support for AJP and load balancing features.Thus, the key differentiators between the two protocols are now the maturity of their load balancing features and the ease with which they can be configured.
As far as ease of configuration is concerned, mod_proxy is the clear winner. The module was developed alongside the Apache Web Server, and its configuration is very straightforward, only requiring a set of changes within Apache Web Server's main configuration file, httpd.conf.
By comparison, mod_jk must be configured within httpd.conf, and then directed to an additional file called workers.properties, which defines all the available Tomcat servers as "workers", as well as a number of "virtual workers", processes that are responsible for the actual work of load balancing. This is often confusing, and can be a real source of frustration. On the other hand, mod_proxy, being the more mature project, offers a much finer-grained level of control over the load balancing.
In terms of sophistication, mod_jk wins hands down, and this makes it our recommended choice if you want real control over your load balancing. Although mod_proxy and mod_jk both include a web GUI, but mod_jk's is much richer, offering a full page of information about each node, as well as a GUI tool for configuring hot load balancing properties, meaning that servers can be taken online and offline for updates one by one without interruption of service.
The load balancing algorithms used by mod_jk are also more robust than mod_proxy's, distributing load based on the number of HTTP sessions per server and each server's "lbfactor", a user-defined value used to incorporate the absolute performance potential of different servers into the equation.
As Apache HTTPD with mod_jk is far and away the most common clustering/load balancing solution used with Tomcat, this is the option we'll talk about later in this article.
Nginx, HAProxy, Wakamole and Spread
Although the official Apache documentation still recommends Apache HTTPD with Tomcat as the standard clustering solution, a growing number of alternative load balancing solutions have begun to gain traction.
These solutions focus around the combination of extremely lightweight, high-performance proxy solutions such as Nginx or HAProxy, which provide load balancing, with Wackamole and Spread, two interoperating open source technologies that provide distributed failover awareness via peer-to-peer messaging and floating IP addresses. With this set-up, it is possible to add multiple load balancer nodes to a cluster, which monitor one another's health, providing an additional layer of failover (Note: Tomcat provides some of this functionality natively via the Tribes component).
Both Nginx and HAProxy are fairly new technologies, with rapidly changing feature sets, so if you're thinking about integrating them into your clusters, it will be a good idea to run some benchmarks and compare their features during your discovery process.
Currently, HAProxy tends to be viewed as the more mature and stable project, but some people use a combination of HAProxy load balancing and Nginx reverse-proxying in an attempt to get the best of both worlds. Using Wakamole and Spread also requires some reading, as the peer to peer and floating IP techniques they employ can cause havoc on a shared network if not properly configured. However, according to those companies that have successfully integrated these solutions into their architectures (Wordpress, for one), the work is worth it. You can find more information about setting up this kind of a system here.
The final piece of the clustering puzzle is session persistence - making sure that the information from an individual user's session is always available to them, even if the server currently hosting their session goes down, so that application state is maintained. There are a number of ways that session persistence can be factored into a cluster.
First of all, factoring the need to run in a clustered environment into the initial spec of an application can influence design decisions to a certain point. Non-complex state information that does not pose a security risk, such as the user's current tab, can be preserved on the client side via hidden fields, cookies, and URI-rewriting. These methods can be used effectively for a variety of data types, but are unsuitable for complex or security-sensitive operations.
Secondly, the majority of modern load balancers, including mod_proxy and mod_jk, support a feature called "session stickiness", which means that the load balancer remembers which cluster worker is storing the session information for each client's request, and proxies all concurrent requests from the same client to the same worker. While this ensures that state is maintained while all servers are working properly, if a server goes down for any reason, while the load balancer will begin directing requests to the remaining active servers, state data stored on the failed server will be lost.
Thus, a method of replicating the server-side session data must be provided to ensure that the cluster will truly never lose a transaction. There are several methods of doing this, which can be combined to create the best performing solution.
The simplest method of replicating session data within a cluster is to copy the data to at least one other worker. This "buddy system" method, in combination with some kind of health check or heartbeat function, allows the load balancer to detect when a server goes offline, and begin passing requests to its appropriate buddy worker.
Ideally, the client should perceive the service as uninterrupted. However, this method can introduce overhead under high loads - the load balancer must preserve increasing amounts of session-routing information, while the Tomcat workers take on database-like load in addition to their dynamic content processing load, which can create a bottleneck.
Load balancer bottleneck can be eliminated by using a multi-cast replication model, where each node of the cluster replicates its session data to every other node. For large environments, this can mean that the overall cluster is split into several smaller clusters using the DeltaManager component. However, small cluster set-ups without significant load should not experience these problems.
Other methods of achieving session persistence are to store the session information in a shared file system or JDBC-compliant database, or to use a cloud-based object cache system such as Terracotta. All of these methods carry an additional performance cost, as they require the additional step of writing and retrieving information to and from a database. However, as the overall goal of clustering is to improve availability, performance, and failover protection, this performance hit must be balanced against the other factors.
An Example Clustered Tomcat Configuration
Tomcat provides fairly sophisticated clustering support, including several kinds of session replication, multicast or unicast heartbeat, cluster division, and more. Much of this configuration can be done right within the same XML configuration files as other Tomcat options, but there are some additional steps and options that can cause trouble.
To help you get started, we've created a step by step guide to Tomcat Clustering, in which we walk you through everything you need to do to get a simple Tomcat cluster with session replication running behind Apache HTTPD. Check it out!