Introduction

This blog post documents the solution that worked for me to encrypt the connection between Apache 2.2.25 and JBoss AS 5.1 on port 8443 in a clustered environment (clustered with mod_cluster 1.2.6 on port 8888), with Apache serving as a proxy (see note below  in this introduction about the network configuration in which Apache serves as this proxy).  The solution was achieved in a Windows environment, but it can, no doubt, be applied to Linux, with appropriate command line and system equivalents. I used a self-signed certificate, so I will include instructions for creating the necessary certificate files.  (I will not include instructions for using a certificate signed by a Certification Authority.)  I will also include in this solution steps to encrypt the connection between the browser client and Apache with SSL and steps to redirect unencrypted requests on port 80 to encrypted requests on port 443.

I spent several weeks attempting to solve encryption between Apache 2.2 and JBoss AS 5.1, partly because I was (and still am) a newbie to both Apache and JBoss, and partly because the documentation I found online was various combinations of inadequate, inscrutable, incorrect, and inapplicable to the version of JBoss I was using.  I do not claim to understand the solution described here at a deep level; what I do know is that it proved to work in my environment, unlike all of the examples in the existing documentation I found online. Hopefully this blog post will be of help to newbie admins who are trying to achieve the same goal instead of become one more contribution to confusion.  I write this blog as a newbie, for newbies, with a specific emphasis on averting the several misconceptions that developed for me as I attempted to interpret the existing online instructions. Apache and JBoss experts may find factual errors in my explanations, and I welcome corrections, but I "do solumnly swear" that the solution, as given in detail here, actually did work in my environment.

I mentioned earlier in this introduction that Apache serves as a proxy in this solution.  In the particular environment for which this solution was implemented, the browser client, the Apache server host, and the JBoss server host are in the same internal network.  For this reason, I cannot vouch for whether the solution exactly as given is appropriate for Apache serving as either a forward proxy or reverse proxy, according to the strict definitions of which both an internal network and the Internet are at play. But even if changes are needed for those topologies, I hope these instructions clear up some misconceptions about Apache and JBoss configuration and set the novice on good conceptual ground for moving forward from here.

These instructions presume a clean installation of both Apache 2.2 and JBoss AS 5.1, with clustering with mod_cluster 1.2.6 already configured. 

Creating the Self-Signed Certificate

Creating a working certificate file set specifically suited to meet the requirements of the Apache 2.2-to-JBoss AS 5.1 SSL connection, as stated by existing online documentation, and knowing the right way to configure the certificate files was the single most difficult and frustrating task involved in the solution.  Had I achieved the right certificate configuration from the outset, it is likely that I would have saved myself days of seemingly endless trial-and-error troubleshooting.  What seemed to be the most promising online instructions talked about creating both a keystore and a truststore for JBoss, but those instructions did not work for me.  It didn't help that the instructions were confusing and incomplete.  So, I hope in this section potentially to save a lot of frustration for developers who find this guide.

These instructions presume that you have a version of the Java JDK installed on your system and that its bin folder is indicated on your system Path variable. (This is for running the keytool commands.)  They also presume that OpenSSL is installed on your system and that its bin folder is indicated on your system Path variable.  For me, OpenSSL came as part of the installation of Apache 2.2.25 I downloaded from http://apache.mirrors.hoobly.com/httpd/binaries/win32/httpd-2.2.25-win32-x86-openssl-0.9.8y.msi (note the "openssl-0.9.8y" part of the MSI file name), but you can install OpenSSL separately. My OpenSSL executable was located under the Apache 2.2 bin folder. Wherever your OpenSSL installation is, the folder in which its executable file exists needs to be in the system Path variable.

1. Create the Certificate and Key Files

Note that in the following command you should change -days to the number of days you want the certificate to be valid. Also note that if you use an installation of OpenSSL that is independent of the Apache installation, the openssl.cnf file will be in a different location from C:\Apache2.2\conf and possibly a different file name.  For example, if you install Win64OpenSSL-0_9_86.exe from the OpenSSL.org site and do not change the default installation directory, the -conf option should point to C:\OpenSSL\bin\openssl.cfg. 

The command below will output a server.crt file, a server.key file, and a .rnd file to the directory from which you run it in a command prompt.  I created a C:\Apache2.2\certs folder and ran this and the subsequent commands in this section from there. I will explain in later sections where these files need to be declared in the Apache and JBoss configurations.

When you run this command, you will be prompted for several items of information.  Fill in information for each item.  The most critical item is the Common Name, which should be the name of the website that will be used in the URLs that go to your application.  For example, if your website is acme.com such that users will enter http://acme.com, then the Common Name should be acme.com.  (Note: if you are configuring this in a local development environment in Windows, you will need to enter a mapping of the local machine's IP address to this website name in your hosts file. See the next section for instructions on how to do this.)

2. Create the Keystore for JBoss

For this solution, only a keystore will be needed.  Much, if not all, of the online documentation I found said to use both a keystore and truststore, but none of the attempts to implement those instructions worked for me. What finally worked was not configuring a truststore and using only a keystore.  I have no doubt that using a truststore would work if properly implemented, but the instructions I found did not provide enough information for me to be successful with that approach.  Hopefully I will provide you with enough information for the keystore-only approach to work for you.

The next command will create a temporary keystore in PKCS12 format from the certificate and key created in the previous step. It will be used to create a keystore in JKS format in the next command.  The command as written presumes that server.crt, server.key, and .rnd are in the same location as the command prompt.  The output keystore will be to the same location.

When you run the following openssl pkcs12 command, you'll be prompted for a password.  In this and many examples online, the password 'changeit' is used. A different password should be used in non-development environments.  These will be the prompts you see when you run the next command.

Note: If your version of Openssl did not create a .rnd file for the –rand option, see alternate commands below.

If there is no .rnd file already in your working directory, then from the command prompt set a RANDFILE system property. This property can be temporary to your command prompt session, so simply running the following command should be sufficient.  Of course, if there is already a valid RANDFILE system property on your machine, this set command does not need to be run.

Now you can run the openssl pkcs12 command without the –rand option to create the keystore.

Next, use the keystore.p12 file to create a new keystore in JKS format. A prerequisite for running this keytool command is an installation of the Java JDK, with its bin folder in the Path system variable. This time you will be prompted to enter a password for the JKS keystore. To avoid complications, use the same password you used for keystore.p12.  You will use the password of the JKS keystore in the JBoss configuration of the JKS keystore, and it helps to keep things straightforward by using the same password for the source and destination keystores when executing the command that follows.  Here is what you will be prompted to enter when you run the command:

The command will output a keystore.jks file to the working directory. After running this command you will have three files that you will use to configure SSL on Apache and JBoss: server.key, server.crt, and keystore.jks. These files will be configured for use in subsequent sections. Run this command.

Configuring SSL between the Browser Client and Apache 2.2

The first SSL connection we'll implement is between the browser and Apache.  The connection will be on port 443, which is the default port for HTTPS. The configuration demonstrated here will include redirecting port 80 (HTTP) to port 443 (HTTPS), but SSL on port 443 is independent of redirection and can be set up without it. The steps as shown in this section are on a clean installation of Apache 2.2.25, so if your installation is not clean you may need to adapt these steps to your configuration context.  On my system Apache is installed at C:\Apache2.2, so I will reference that location in the instructions, but you should substitute your own location in for it.

1. 1. Configure Port 443 with SSL

Open the httpd.conf file that is found under C:\Apache2.2\conf in a text editor of your choice. Find the line that reads "LoadModule ssl_module modules/mod_ssl.so" and uncomment it (that is, remove the '#' from the beginning of the line). In context, the line should look like the middle one here:

In the same file, find the line that reads "Include conf/extra/httpd-ssl.conf" and uncomment it, too. In context, it should look like this.

The above Include directive tells the Apache compiler to include the directives that are stated in the httpd-ssl.conf file. So, now open that file from C:\Apache2.2\conf\extra in a text editor.  Find the line that reads " <VirtualHost _default_:443>". After this line you will see several directives, one of which is ServerName.  Apache programmatically set this value to the computer name on which it was installed, followed by port number 443.  For example, if the full computer name is mycomputer.acme.lcl, the Apache installation will have automatically set this line to read

ServerName mycomputer.acme.lcl:443

If this name does not match the Common Name you used when you created the certificate in the previous section, then you will need to change the ServerName value to use the Common Name by which you identified the certificate.  In this tutorial, we used acme.com, so ServerName should be changed to

ServerName acme.com:443

As stated previously, in a Windows system for local development you will need to enter a mapping for the Common Name in your hosts file. This file can be found in %WINDIR%\System32\drivers\etc.  The text editor used to modify it must be opened as an administrator.  After substituting your IP address and Common Name, the edited hosts file should contain the new entry, as shown in the next box.  The IP address should be mapped only once.  If you map the same IP address to multiple domain names, JBoss will be able to read only the first one, and if it doesn't match the name in the certificate, the SSL connection from Apache to JBoss will fail.  If you are not in a development environment and your web domain is already associated with the host IP through DNS (as is usually the case with system test servers, for example, and is always the case with production servers), then you do not need to modify the hosts file.

Note, as shown below, that for the ErrorLog and TransferLog directives in the httpd-ssl.conf file, I have modified the log file names to 443error.log and 443access.log, respectively. This is not necessary, but it helps distinguish between logs for the port 443 virtual host and logs for other virtual hosts.  (For example, if you already have Apache and JBoss configured in a clustered environment--a condition that this tutorial presumes--then you most likely have a virtual host declared in your httpd.conf file for the mod_cluster_manager.  In my configuration the mod_cluster_manager virtual host is declared to be on port 8888, so it has been useful to distinguish between port 443 and port 8888 error and access logs.)  Also note that the Apache installation created an admin email address based on the computer domain extension: admin@acme.lcl. You should change this to an appropriate email address, such as yourname@acme.com. In context, assuming you followed the changes explained above, the lines should look like what is in the following example.

IMPORTANT: All paths in Apache .conf files should be created with forward slashes, not back slashes, since the latter are not recognized by the compiler. 

Note that immediately below these lines is a directive

SSLEngine on

This is critical to a successful SSL connection and should be left uncommented.  Further below this line, find the directives SSLCertificateFile and SSLCertificateKeyFile.  Change the former to point to the server.crt file and the latter to point to the server.key file created in the previous section. In my case, I created these files in C:\Apache2.2\certs, so I will point to these files in that location. (Alternately, I could move the certificate and key files to C:\Apache2.2\conf, which is the preset location.)  After these changes, the directives will look like this:

Quick Check

After saving all edited files, restart the Apache service. This action needs to be taken as an administrator, so you need credentials to an adminstrator account on the machine. On Windows 7, for example (assuming you are prototyping the configuration in a development environment), you can access the Services console through Windows Start > type "services" in the search box > right-click on Services when it appears from the search > click "Run as administator".  You will be prompted for administrator credentials if you are not already logged in as an administrator.  When the Services console opens, highlight the Apache2.2 service in the list and click the Restart link.

After the Apache service has restarted, open your browser and, substituting your application domain, enter https://acme.com and press Enter.  Since the SSL configuration uses the self-signed certificate you created, the browser will prompt you for a response to the untrusted certificate.  If you are using Internet Explorer the option link to accept the certificate reads "Continue to this website (not recommended)".  Click this link.  If the configuration has no issues, the default Apache page that reads "It works!" will load through the HTTPS connection on port 443.  After the page loads, notice that the URL remains https://acme.com.

2. Configure Redirection from Port 80 to Port 443

At this point, if you enter the non-SSL URL http://acme.com, the "It works!" page will load over the unencrypted HTTP connection on port 80.  If you want to redirect this URL to HTTPS on port 443, perform the following steps.  Note that this redirection is not critical, but not configuring it will allow for unencrypted connections to your application.  That may be appropriate for some pages of your application, in which case you'll need to refer to other documentation for how to configure some pages to load over port 80 (HTTP) and others to load over port 443 (HTTPS). But for the purposes of this tutorial, the assumption is that all resources from your application will travel through encrypted connections.

There are many directive variations of redirection, which fall into two categories.  The first category is the responsibility of the module mod_alias, and the second category is the responsibility of the module mod_rewrite.  In general, mod_alias supports simple redirection and has less performance cost, while mod_rewrite is relatively more expensive performance-wise but supports more complex redirection rules.  For the purposes of this tutorial, I will limit the configuration to the redirection of the server root to port 443 and use the RedirectMatch directive from mod_alias. That is,  I will demonstrate only redirection from http://acme.com to https://acme.com. Note that this will assume users access your app directly at the server root. If, however, they needed to access it through a separate application root--for example, acmeApp--you would need to modify the RedirectMatch directive to the appropriate URL; for example,

RedirectMatch ^/$ https://acme.com/acmeApp/

instead of what is shown in the next box.  This would redirect from http://acme.com to https://acme.com/acmeApp. See the official Apache mod_alias and mod_rewrite documentation for more complex redirection capabilities.

In the httpd.conf file, which you edited earlier, find the line that reads "Listen 80".  In a clean installation, this line will be uncommented by default; it should remain uncommented for this tutorial. Directly underneath this line, add a VirtualHost for port 80, as shown below.  You will need to substitute your IP address and the server name in both the ServerName and the RedirectMatch directives.  These values should match those that you entered into your Windows system's hosts file in the previous subsection. It is critical that ":80" not be removed from the VirtuaHost declaration and that the redirection be to the HTTPS version of your application's URL. The "Match" part of RedirectMatch allows for Perl compatible regular expressions to be used, and in this configuration, the regular expression operators ^ and $ are being used to limit redirection to the server root. The / character in the substring ^/$ represents the server root. See documentation on Perl Compatible Regular Expressions for more information about the meaning of ^, $ and other operators, and how to use them.

Note that the RedirectMatch directive requires that the mod_alias module be loaded, which is done when the line "LoadModule alias_module modules/mod_alias.so" in the httpd.conf file is uncommented.  This line is uncommented by default in a fresh installation, but if it has been commented in your Apache instance, you need to uncomment it before restarting with the RedirectMatch directive in place, as shown here:

Quick Check

Restart the Apache service again (as described above). Enter http://acme.com (substituting your Common Name).  If all is well, you should be prompted to trust the self-signed certificate again, and when you click "Contine to this website (not recommended)" the "It works!" page will load through HTTPS on port 443.  Once the page is loaded, the displayed URL should read https://acme.com.

Configuring SSL between Apache 2.2 and JBoss AS 5.1

Now comes what was for me the hard part, which I anticipate will be easy for you as you follow this guide.  I won't get into all the points of confusion that I had to struggle with, but if you are a newbie and have already tried some of the documents I tried to use, you are probably familiar enough with them. In the end, the configuration that worked for me, and which I document here, is simple, but it took me several weeks to tangle and untangle the misconceptions that grew in my mind as I tried as a newbie to interpret the various online documents I found for this task.

As stated earlier, this section presumes that you have Apache 2.2 and JBoss AS 5.1 already configured in a cluster with Apache's mod_cluster 1.2.6 (or 1.2.0; see note below), with Apache on the front end as the access point and at least one JBoss server as a cluster node.  In this tutorial, the 'all' server will be used as the only node (found at JBOSS_HOME\server\all, where JBOSS_HOME represents the home installation directory of your JBoss instance). If you have multiple nodes in your configuration, you simply apply the same steps for editing server.xml on each node, as I describe below for the one node in my tutorial configuration. 

I use mod_cluster 1.2.6 here, but I also successfully configured the SSL connection between Apache and JBoss in a cluster configured with mod_cluster 1.2.0.  However, the 1.2.0 version of mod_cluster did not support the use of the RewriteRule directive (which I mentioned above but do not demonstrate here), so I upgraded to mod_cluster 1.2.6.

1. Configure Apache for SSL Proxying to JBoss

Open the httpd-ssl.conf file that you edited in the previous section. This is located at APACHE_HOME\conf\extras\httpd.ssl in a standard Apache 2.2 installation, where APACHE_HOME represents the root directory of your installation.  Find the virtual host for port 443 in this file.  The default declaration is "<VirtualHost _default_:443>", but if it has been modified in your configuration the "_default_" portion might have been replaced with your host IP address.  Assuming that you have already configured the ServerName, SSLEngine, SSLCertificateFile, and SSLCertificateKeyFile directives as described in the previous section of this tutorial, add the following directives somewhere after the ServerName directive, substituting values particular to your environment.  I will explain the directives and which values to use, below.

SSLProxyEngine On

SSLProxyCACertificateFile "/Apache22/certs/server.crt"

ProxyPass / https://192.103.69.99:8443/

ProxyPassReverse / https://192.103.69.99:8443/

In my httpd-ssl.conf file they are situated between the TransferLog and SSLEngine directives.

In the configuration shown, requests to the Apache server on port 443 are being redirected to port 8443 on the JBoss server.  In the next step, the 'all' JBoss server, which is the only node in my cluster, will be configured to accept requests on port 8443.  The two directives ProxyPass and ProxyPassReverse are where you configure requests on port 443 to "pass through" Apache and go on to JBoss on port 8443. Paraphrasing, ProxyPass means something like, "Apache is a proxy that incoming requests pass through", and ProxyPassReverse means something like, "Returning responses from JBoss can pass back through Apache to the browser." A single SSL session is being established for round trip communication between the browser and JBoss, with Apache in the middle.  The first segment of the session channel is over port 443 between the browser and Apache, and the second segment of the session channel is over port 8443 between Apache and JBoss.

Note that the only modifications you need to perform on the Apache configuration for SSL connection to JBoss is in this virtual host for port 443.  Assuming a standard mod_cluster configuration, there is a separate virtual host, probably declared in your httpd.conf file, or perhaps in APACHE_HOST\conf\extras\httpd-vhosts.conf, which is used for managing the cluster, but this virtual host should not be configured to use SSL.  In my configuration the port of this virtual host is 8888, but on yours it most likely is some other number.  Depending on how your cluster is configured, you may be able to identify this virtual host by one of two ways. The first way is to find a virtual host in which the ManagerBalancerName directive is declared (probably in either the httpd.conf or httpd-vhosts.conf file). 

The second way is to locate the file JBOSS_HOME\server\all\deploy\mod_cluster.sar\META-INF\mod_cluster-jboss-beans.xml. Depending on which server your configuration uses, the "all" portion of this path may be something different; it depends on which server you are using. For example, if you are using the 'default' server, the path would be JBOSS_HOME\server\default\deploy\mod_cluster.sar\META-INF\mod_cluster-jboss-beans.xml.  It could also be a custom server name defined by a subdirectory of JBOSS_HOME\server; for example, JBOSS_HOME\server\node1, where node1 is a custom server that constitutes the first node of your cluster.  Upon locating the mod_cluster-jboss-beans.xml file--there should be one such named file for each server node in your cluster--find the proxyList property. For each Apache server in the cluster, there may be a hard-coded IP address and port number (but there might not, in which case this method won't work for discovering which port your mod_cluster_manager is on).  For example, in my configuration, the proxyList property looks like this:

This shows that cluster management is occurring over port 8888, and there should be a VirtualHost declared for this port in httpd.conf, or perhaps in APACHE_HOST\conf\extras\httpd-vhosts.conf.  The bottom line is that this virtual host should not be configured for SSL.  If your cluster is already working before you attempt to configure SSL between Apache and JBoss, you should not modify any of the directives within this virtual host. The application resource traffic between Apache and JBoss will be over port 8443, so 8443 is the port that will be encrypted (in the step below).  The mod_cluster_manager port (in my case, port 8888) should be left alone for the purposes of this tutorial. I spent a lot of time confused about which port to encrypt; that's why I've gone to lengths to make this point clear.

2. Configure JBoss Nodes for SSL Connectivity to Apache

Now the JBoss server needs to be configured to accept application traffic over port 8443.  In the previous section, we configured Apache to redirect requests it receives on it's own port 443 to the JBoss host's port 8443. But for that to work, we must configure JBoss to accept encrypted application requests on port 8443.

First, on the JBoss server--if you are configuring in a development environment, this may be the same server on which Apache is running--copy the keystore.jks file you created earlier in this tutorial to a location under your JBoss installation. In my case, I created a new folder C:\JBoss_AS_5.1\server\all\certs and pasted the keystore file there.  C:\JBoss_AS_5.1 is the home directory of my JBoss installation.  I'll refer to this location in subsequent steps as JBOSS_HOME.

Next, open the file JBOSS_HOME\server\all\deploy\jbossweb.sar\server.xml in a text editor of your choice. As stated earlier, the "all" portion of this path will be different if you are using a server other than the 'all' server.  Uncomment the Connector element for port 8443 and edit it as shown below, making appropriate modifications for the location of the keystore.jks file you pasted above and the password you used when you created it.  Note that there will also be Connector elements for ports 8009 and 8080.  These should be left alone.

Pay special attention to the sslProtocol attribute value.  This needs to be within the scope of the SSLProtocol directive that is declared in the httpd-ssl.conf file on the Apache server.  In my case, that directive is declared as

This directive accepts all SSL protocols except SSLv2.  As of this writing TLS is the most secure protocol, but if you declare this protocol you may need to modify a property on your browser to enable it. In the default installation, JBoss sets TLS as the value of the sslProtocol attribute, as shown below. For the purposes of this tutorial, I will use TLS.  In your case, you may want either to change this to SSLv3 or to stay with TLS, making sure that your browser settings accept it and that it is within the declared scope of the SSLProtocol directive in httpd-ssl.conf.  (To enable TLS in Internet Explorer, go to Tools > Internet Options > Advanced tab > scroll down to Security > and check the "Use TLS x.x" options you want to use.)  Note that the clientAuth attribute is set to false.  It may be tempting to change this to true, but it needs to be left at false for the connection to succeed because setting it to true means that the SSL handshake will require a valid certificate chain, which is not used with a self-signed certificate.

Here is what the Connector element in server.xml should look like:

Quick Check

Restart the JBoss server.  Depending on your configuration, you may be able to do this through the Windows Services console or by starting JBoss from the command line using JBOSS_HOME\bin\run.bat. Now send a request to JBoss over port 8443 directly, without going through Apache. For example, if the IP of your JBoss host is 192.103.69.99, send this URL:

     https://192.103.69.99:8443

Whatever resource is configured to load from the server root should display over HTTPS after you accept the unsigned certificate by clicking "Continue to this website (not recommended)" on Internet Explorer. This might by your application's home page, or it might be the JBoss management page, depending on your configuration.

Final Check

If the connection directly from the browser to JBoss over port 8443 succeeded in the previous quick check, you know that JBoss is configured to accept proxy connections from Apache on that port. When Apache was configured to serve as a proxy to JBoss in step 1 of this section, ProxyPass was set to forward requests over port 443 to JBoss over port 8443. And earlier, port 80 was configured to redirect requests for the resource at the server root to port 443.  If all is well, then, a request from the browser for the server root over port 80 (that is, using http://) should redirect to the Apache host's port 443, which will forward the request to JBoss's port 8443. Here's the final test:

     http://acme.com

If I have written these instructions correctly and clearly enough, you have implemented them as described, and there are no complicating factors in your environment that weren't in mine, you should be prompted to trust the self-signed certificate, and the expected resource should load from the JBoss server root over an SSL connection.