Skip to end of metadata
Go to start of metadata

Since it's just taken me 4 days to get a working setup where JASPA will talk via a load balancer to multiple AM instances, I thought I'd better document what I've done.  I started off following Rich Riley's guide OpenAM Clustering Setup but there are many pitfalls waiting which I'd like to avoid in the future.  Note that with all my Agent/AM setups, I put the AM instances (and the load balancer) in one domain, and the Agent in an entirely different domain.  This works best since CDSSO is enabled by default now.  You can get things working in the same domain, but the following steps assume different domains.  Putting your Agent in a separate domain is very simple, just add a new domain entry into /etc/hosts and use that everywhere you refer to the agent.

You must do these steps in order, if you try (for example) to configure users and groups in OpenAM instance #1 before configuring instance #3, then instance #3 configuration WILL FAIL.

Deploy N instances of OpenAM into N instances of Tomcat

I'm assuming here that you are familiar with each of the painful steps involved in doing this, so I'm going to gloss over them.  Rich's document deals with this in more detail, although note that you will need to use Tomcat 8+ for your OpenAM containers in order for Websocket Notifications to work.  JASPA is less fussy about the version of its container and will accept any old rubbish.

Download and deploy OpenDJ

Download OpenDJ from here and unpack it somewhere permanent (i.e. not /tmp).  If you find you already OpenDJ installed, you are advised to remove it by running the uninstall script in the top level directory.  I recommend OpenDJ version 3 upwards.  Once you've downloaded and unpacked, run setup in the top level directory.  This fires up a cute graphical installer.  About the only things to do during setup are:

  1. Set a password ("password" is quite sufficient, although you can choose to be more exotic).
  2. Make a note of the port number.  This will most likely be 1389.
  3. Set the base DN for data to dc=openam,dc=forgerock,dc=org, i.e. definitely not what OpenDJ suggests.
  4. Under "Directory Data", click "Only Create Base Entry".

Download and deploy haproxy

Download haproxy from the interweb and install it somewhere permanent. One way to install haproxy is to use brew (sudo brew install haproxy). 

I have no idea how the default haproxy.cfg looks, but it will almost certainly be useless.  The one I use is as follows.  You will need to edit OPENAM_URL_1, OPENAM_URL_2, etc. etc. accordingly.  Each will need to be the full path (excluding scheme) to one of your OpenAM instances, e.g.: openam.example.com:8010/openam.  Note that my config file puts haproxy onto port 8080, which will cramp your style if you have a Tomcat running there.  I have my Tomcats running on 8010, 8020, 8030, etc.   So far I don't have 8 of them.  So far.

Command to start haproxy is `haproxy -f haproxy.cfg`.

The following config file may give WARNING messages if you are using latest haproxy, that is because some of the directives listed below are now deprecated.

# this config needs haproxy-1.1.28 or haproxy-1.2.1

global
        log 127.0.0.1   local0
        log 127.0.0.1   local1 notice
        maxconn 4096
        user root
        group wheel

defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        retries 3
        option redispatch
        maxconn 2000
        contimeout      5000
        clitimeout      50000
        srvtimeout      50000

frontend http-farm
  bind *:8080
  default_backend openam

backend openam
  balance roundrobin
  cookie amlbcookie indirect nocache  
  server openam1 OPENAM_URL_1 cookie 01 
  server openam2 OPENAM_URL_2 cookie 02
  server openam3 OPENAM_URL_3 cookie 03

Wipe out any and all OpenAM configuration on your machine

While all your Tomcat instances are dead, you'll need to remove any and all previous OpenAM configuration.  The directory $HOME/.openamcfg will contain one file per OpenAM instance, each containing the name of one directory.  Each of these directories will need to be removed, followed by the $HOME/.openamcfg directory itself.

Configure the first OpenAM instance

Start up the Tomcat container for one of your OpenAM instances, it doesn't matter which one.  In a browser, navigate to that OpenAM instance and select Create New Configuration

  • At Step 2 "Server Settings", choose a configuration directory which will be unique to this instance.  I usually make this the equivalent of $HOME/openamN where N is the instance number
  • At Step 3 "Configuration Data Store Settings" you need to choose "First instance" and leave all the settings alone (configuration store is OpenAM and NOT OpenDJ)
  • At Step 4 "User Data Store Settings" you should select "Other User Data Store" and "OpenDJ"
    • Specify the port number you noted down when installing DJ
    • Specify the password you chose when installing DJ
  • At Step 5 "Site Configuration" you must
    • Select "Yes"
    • Choose a site name - any name, it doesn't have to exist - the setup will create it for you.  I chose main
    • Enter the URL of haproxy.  This will be http://YOUR-DOMAIN:BINDING-PORT/BACKEND
      • You'll have to sort out YOUR-DOMAIN (mine is openam.example.com).  Although I guess "localhost" would do, I didn't use it.
      • BINDING-PORT is the one you specified under frontend > bind in the config file above (i.e. 8080)
      • BACKEND is the "openam" thing in the config file above
      • This makes my Load Balancer URL: http://openam.example.com:8080/openam
  • Race through the remaining step and onwards to victory

Configure the remaining OpenAM instances

For each of the remaining instances, start up your Tomcat container and browse to OpenAM running in that instance and again select Create New Configuration

  • At Step 3 "Configuration Data Store Settings"
    • Select "Add to Existing Deployment"
    • Enter the URL of your first OpenAM instance (not the load balancer)
  • Step 4 will be skipped
  • At Step 5 "Site Configuration"
    • Select "Yes"
    • Enter the site name you gave when setting up the first instance (main?) and the load balancer URL
  • Race onwards towards victory

Setup users, groups, agents and OpenDJ

It is about this point that I startup one of the OpenAM instances (it actually doesn't matter which one) and start setting up users and agents in CDSSO mode.

  1. Top Level Realm > Data Stores > OpenDJ:  Click "Load schema when saved" and click Save.  This will create the schema within OpenDJ.
  2. Top Level Realm > Subjects > Groups: I usually add a group called "Accessors".
  3. Users: Add at least two users, one a member of group Accessors and another who isn't.  Note that you won't get the demo user for free.
  4. Top Level Realm > Agents > JEE: Create a J2EE agent with an appropriate password.  Don't forget the Server URL is the load balancer URL, i.e. http://openam.example.com:8080/openam
  5. Global: Agent Root URL for CDSSO: Add the load balancer URL to the existing JASPA URL, i.e. http://openam.example.com:8080/ NOTE: trailing "/" is important!
  6. Global: General: Agent Debug Level: Set to message
  7. Don't forget to save.
  8. SSO: Cross Domain SSO: ensure the correct load balancer path is in the CDSSO Servlet URL (it will have cdcservlet stuck on the end of it).
  9. SSO: CDSSO Trusted ID Provider: Remove the URL containing the load balancer path and add the cdcservlet URL for EACH ONE of your OpenAM instances (this is necessary for LARES to work with Agent 3.x)
  10. SSO: CDSSO Domain List: Add the JASPA domain.
  11. Don't forget to save.
  12. Top Level Realm > Authorization > Policy Sets> Default Policy Set: Add a policy specifying that only "accessors" have access.  The policy URI can just be the default with all the stars in it.

Configure the JASPA instance

Deploy JASPA into your nominated Tomcat instance.  You probably already know that OpenAM and JASPA don't play well in the same container, so choose a different container to the OpenAM ones. The most critical point here is that you MUST specify the URL of the load balancer when prompted for the OpenAM URL.  If you mistakenly type the URL of one of the AM instances, the agent will die on startup with the following logged to catalina.out:

com.sun.identity.agents.arch.AgentException: ApplicationSSOTokenProvider.getApplicationSSOToken(): Unable to get Application SSO Token
 at com.sun.identity.agents.common.ApplicationSSOTokenProvider.getApplicationSSOToken(ApplicationSSOTokenProvider.java:81)
 at com.sun.identity.agents.arch.AgentConfiguration.setAppSSOToken(AgentConfiguration.java:616)
 at com.sun.identity.agents.arch.AgentConfiguration.bootStrapClientConfiguration(AgentConfiguration.java:722)
 at com.sun.identity.agents.arch.AgentConfiguration.initializeConfiguration(AgentConfiguration.java:1140)
 at com.sun.identity.agents.arch.AgentConfiguration.<clinit>(AgentConfiguration.java:1579)
 at com.sun.identity.agents.arch.Manager.<clinit>(Manager.java:675)
 at com.sun.identity.agents.tomcat.v6.AmTomcatRealm.<clinit>(AmTomcatRealm.java:67)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at java.lang.Class.newInstance(Class.java:442)
 at org.apache.tomcat.util.digester.ObjectCreateRule.begin(ObjectCreateRule.java:145)
 at org.apache.tomcat.util.digester.Digester.startElement(Digester.java:1282)
 at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509)
 at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:182)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1338)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2781)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:504)
 at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
 at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
 at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
 at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
 at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
 at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1555)
 at org.apache.catalina.startup.Catalina.load(Catalina.java:615)
 at org.apache.catalina.startup.Catalina.load(Catalina.java:663)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:280)
 at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:454)
--------
com.sun.identity.authentication.spi.AuthLoginException: Failed to create new Authentication Context: Cannot find server ID.
 at com.sun.identity.authentication.AuthContext.login(AuthContext.java:691)
 at com.sun.identity.authentication.AuthContext.login(AuthContext.java:614)
 at com.sun.identity.authentication.AuthContext.login(AuthContext.java:417)
 at com.sun.identity.agents.common.ApplicationSSOTokenProvider.getApplicationSSOToken(ApplicationSSOTokenProvider.java:64)
 at com.sun.identity.agents.arch.AgentConfiguration.setAppSSOToken(AgentConfiguration.java:616)
 at com.sun.identity.agents.arch.AgentConfiguration.bootStrapClientConfiguration(AgentConfiguration.java:722)
 at com.sun.identity.agents.arch.AgentConfiguration.initializeConfiguration(AgentConfiguration.java:1140)
 at com.sun.identity.agents.arch.AgentConfiguration.<clinit>(AgentConfiguration.java:1579)
 at com.sun.identity.agents.arch.Manager.<clinit>(Manager.java:675)
 at com.sun.identity.agents.tomcat.v6.AmTomcatRealm.<clinit>(AmTomcatRealm.java:67)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at java.lang.Class.newInstance(Class.java:442)
 at org.apache.tomcat.util.digester.ObjectCreateRule.begin(ObjectCreateRule.java:145)
 at org.apache.tomcat.util.digester.Digester.startElement(Digester.java:1282)
 at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509)
 at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:182)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1338)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2781)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:504)
 at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
 at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
 at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
 at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
 at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
 at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1555)
 at org.apache.catalina.startup.Catalina.load(Catalina.java:615)
 at org.apache.catalina.startup.Catalina.load(Catalina.java:663)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:280)
 at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:454)

 

What to look for

Watch to see if any of the Tomcat or OpenAM instances die on startup.  By checking the catalina.log you will be able to see if there are any binding errors.  This will indicate you have two or more of your Tomcat instances trying to use the same port.  Unfortunately there is no easy way to configure this.

If you're using JASPA 3.x, then if you haven't configured OpenAM correctly it will die on startup ("cannot obtain application SSO token", or something similar).  JASPA 5 may survive this, but will give "access denied" to any incoming request.

 

 

 

 

 

  • No labels