3 Replies Latest reply on Jan 15, 2013 11:09 PM by haswellj

    Java EE Application Structure for 'always on' data gathering

    mbelling

      One of the projects I am working on will be receiving data from an external source, and constantly processing that data.  The plan is to build on top of Java EE/JBoss that application, and an application for accessing the information that is generated from that external data. From the information I have found so far on Java EE seems to be a 'trigger/transaction based' approach, where either a client connection, or specific message would drive processing in a Java EE application.

       

      Is there a standard Java EE way to have an 'always on' process that gathers data?  Or is this something I would develop outside Java EE and just have my Java EE App connect to it when someone needs data from it?  Is there any documentation on this sort of approach?

       

      Thanks,

      ~Matt

        • 1. Re: Java EE Application Structure for 'always on' data gathering
          haswellj

          Hi Matt,

           

          I'm not really sure I fully understand the question, but I suspect the answer is "it depends."  What sort of data are you gathering? How are you gathering it?  By "always on" do you mean you have some service that you need hard guarantees as to its availability?  I would probably begin by identifying what your application needs to do exactly, then posing questions about the specifics as you encounter difficulties in implementing it.  You might look into SOA or ROA and the relevant Java technologies (e.g. Switchyard) to see if those address your needs.

           

          Best!

           

           

          Joe

          • 2. Re: Java EE Application Structure for 'always on' data gathering
            mbelling

            Hi Joe,

             

            To add more detail: I will be receiving seperate xml data types over several JMS connections.  By always on, I mean that I will constantly be connected to and listening on the JMS connections for messages. Those messages will get distibuted to different modules for processing, and the results from each module will be cached/stored and updates will be sent to connected clients.  The cache/storage will be there for any new client that connects, which will need the full 'state'.

             

            I am an experienced Java developer, but up to this point I have only been doing JSVC and Apache Felix OSGi processes.  I am unfamiliar with Java EE, and am not even sure if my 'problem' fits in the Java EE solution set.  I was given this task and told that I have 'free reign' to look in to new technologies that would facilitate high data load, clustering, high availability, etc.  Hence my questions here.

             

            Thank you for any insight you may provide,

            ~Matt

            • 3. Re: Java EE Application Structure for 'always on' data gathering
              haswellj

              Hi Matt,

               

              Sorry about the delay.  I have worked on similar projects, and JEE/JMS works well for this sort of thing.  Another technology you might consider is Akka, which could easily help you implement the message distribution.  JEE is a good candidate for this sort of project if you also need things like persistence, web capabilities, JNDI, etc.  Obviously, most JEE containers implement clustering and failover, too, which is a big mark in their favor.  For very high message volumes, you always need to get creative with caching, load-balancing, batching, etc., but in my experience, JEE scales very well out of the box (to, depending on the task and configuration, 10s to 100s of thousands of operations/second)

               

              Best!

               

              Josiah