1 2 Previous Next 29 Replies Latest reply: Sep 24, 2012 12:15 PM by mazz Go to original post RSS
  • 16. Re: Does HornetQ support JBBC persistence
    Yong Hao Gao Master

    doh!

     

    OK I'd like to take on this one.

     

    Howard

  • 17. Re: Does HornetQ support JBBC persistence
    Justin Bertram Master

    Another thing to think about is that once we have implemented message replication between nodes you will no longer need a shared store for high-availability of messages so no SAN will be necessary.

  • 18. Re: Does HornetQ support JBBC persistence
    Yong Hao Gao Master

    That's true. I think people choose DB for persitent store because they are either very familiar with it or they already using it for their many other applications for a long time. Already there are many rich tools for the user to explore the tables and modify the records in it. With our native journals the data is more obscure and the tool is primitive and not standard.

  • 19. Re: Does HornetQ support JBBC persistence
    Andy Taylor Master

    JDBC persistence is not something we will support, The JIRA is their in case anyone from the community wants to implement a JDBC journal manager. AS Justin says, replication will be available soon for any one who cant use a SAN.

  • 20. Re: Does HornetQ support JBBC persistence
    Yong Hao Gao Master

    ok, I'll take it as a low priority.

  • 21. Re: Does HornetQ support JBBC persistence
    Justin Bertram Master

    In my opinion, if our tools are primitive then we should invest in writing better tools.  A while back Clebert asked me to do an XML import/export journal tool which should be a bit more user-friendly than the traditional PrintData (see HORNETQ-787).  I'm sure there's other areas where we can improve our tooling.  Did you have any specific ideas or points of concern, Howard?

  • 22. Re: Does HornetQ support JBBC persistence
    Andy Taylor Master

    agreed, we should expose some of the print tool output via some management API so we can expose it either via our own small gui or thru the AS web/admin consoles

  • 23. Re: Does HornetQ support JBBC persistence
    Yong Hao Gao Master

    Well yeah, I think we should have a command-line tool at least, something like the one in AS7, or mysql of MySQL, so we can use simple commands to explore the journal, like

     

    to pick up a specific journal:

     

    > use <journal dir> (or journal name)

     

    > list

    > update

    > delete

    > export

    > import

    > add (insert)

     

    The purpose should be for users/developers to do some low level debugs.

     

    Howard

  • 24. Re: Does HornetQ support JBBC persistence
    source dev Newbie

    Agree that HQ not supporting JDBC out of the box is a big issue for us too.

     

    Again, we're not after performance but, like many other large enterprises, our infrastructure is focussed around a hardened database layer - backups, cross-site I/O, distributed file systems et al.   We do not wish to replicate this for our application layer - installation of new disk capacity, backups, and beefed up cross-site links in the app layer has a considerable cost implication and we already have a perfectly good, hardened db layer. 

     

    So we are stuck at JBoss EAP 5, unless we implement a JDBC journal or use JBoss 6 with an alternative messaging provider.  Anyone done either of these?

     

    Cheers.

  • 25. Re: Does HornetQ support JBBC persistence
    Justin Bertram Master

    Once replication is fully implemented (the initial implementation is already available in 2.3.0.Alpha) I think your use-case will be pretty much covered.

     

    It's worth noting that lots of applications are moving away from the RDBMS (witness the rise of no-sql).  Organizations which have built a large infrastructure on the RDBMS will not be able to adapt quickly and are at risk of being left behind in certain areas.  Of course, such is life in IT.

  • 26. Re: Does HornetQ support JBBC persistence
    source dev Newbie

    Justin, thanks for the response. 

     

    The functional use cases for resiliancy may be covered but, unfortunately for us, that's not the whole story by a long shot.  As you say, many organisations have built around the database (and still do, despite no-sql, which is certainly not a panacea for RDBMS), and since that includes things such as off-site backups and the disk space to house massive amounts of storage (including that required for large volume messaging systems), it would take some time to turn that around, even if they wanted to. I'm not convinced they will all share your views since they would have to worry about data loss in multiple layers of the platform, and usually that involves complexity.  My feeling is that HornetQ is less likely to be the solution in those cases since a) a single software component is generally easier to switch out than an infrastructure revamp, especially in this 'pluggable age' and b) many organisations do not wish to revamp their entire infrastructure to pgrade a single software component c) data resiliency complexity being brought into the next layer up (anyone that has tried to configure JBoss 4.3 EAP out of the box for decent transaction recovery will know that such features usually require a lot of configuration management and head scratching).

     

    In our case, through experience, it takes around a couple of months to swap out to a new messaging product (ActiveMQ->OpenMQ->JBM over the years so far - each time for support/resiliancy issues), but any of those would still be cheaper than the infrastructure revamp we need to accomodate HQ.  Of course, I like the idea of getting a bundled messaging component and the integration that comes with it (although admittedly, JBM could have been better integrated in JBoss EAP - shame), but a change to infrastructure for a single software component isn't an option for us right now, especially since they key feature (speed) is not important to us (again, we are looking to jump from JBM because of support/resiliancy issues).

     

    So I suspect the biggest losers here will be the 'upgraders' - existing JBoss customers already using JBM with an RDBMS that will soon run out of road, due to support ending and face significant infrastrcuture costs to leap to the next version.  Unfortuntely, that's my camp. 

  • 27. Re: Does HornetQ support JBBC persistence
    Andy Taylor Master

    The functional use cases for resiliancy may be covered but, unfortunately for us, that's not the whole story by a long shot.  As you say, many organisations have built around the database (and still do, despite no-sql, which is certainly not a panacea for RDBMS), and since that includes things such as off-site backups and the disk space to house massive amounts of storage (including that required for large volume messaging systems), it would take some time to turn that around, even if they wanted to. I'm not convinced they will all share your views since they would have to worry about data loss in multiple layers of the platform, and usually that involves complexity.  My feeling is that HornetQ is less likely to be the solution in those cases since a) a single software component is generally easier to switch out than an infrastructure revamp, especially in this 'pluggable age' and b) many organisations do not wish to revamp their entire infrastructure to pgrade a single software component c) data resiliency complexity being brought into the next layer up (anyone that has tried to configure JBoss 4.3 EAP out of the box for decent transaction recovery will know that such features usually require a lot of configuration management and head scratching).

    Lots of Customers have already share Justin's view and moved away from databases, we have many customers already who have complex HA scenarios and handle massive amounts of data who have made the change. Most messaging Vendors have moved away from  RDBMS as their primary form of Persistence and altho some do have database  implementations they discourage the use of them. Granted in your case this may not be so but in General once users/IT admin etc understand the advantages and disadvantages we are seeing that they are happy to migrate.

     

    Saying that, remember HornetQ is open source and implementing  the persistance layer to use a database would be pretty simplistic (famous last words) ,its just we dont see the need or demand  to do it, but that doesnt' mean that someone couldn't provide an implementation.

     

    One other quick point its not just performance thats the key feature but scalibilty as well.

     

    A quick note on your comment about tx recovery configuration, yes in 4.x it was a head scratcher but in AS7 this is now pretty transparent with resources registering with the recovery manager automatically, this includes HornetQ.

  • 28. Re: Does HornetQ support JBBC persistence
    Justin Bertram Master

    I didn't mean to suggest that no-sql was some kind of panacea for the traditional RDBMS.  I only meant to highlight the fact that the RDBMS is itself not a panacea for data storage and access (which is often an implicit assertion in organizations which build a monolithic infrastructure around the RDBMS).  In fact, I don't believe in panaceas of any kind, but I digress.

     

    As you note, data access and storage across the application tier can become complex and having all that data in the same place (e.g. an RDBMS) can be convenient for certain uses cases, but we haven't yet found that set of use-cases compelling enough to implement a JDBC persistence mechanism.  Although, as Andy noted this is open for the community.

     

    To your point about other organizations which are in your position not all agreeing with the position I outlined above, I think that's OK.  I personally feel that the time and effort put forth into our current persistence implementation (in lieu of a JDBC mechanism) has provided more opportunities for HornetQ than it has cut off.  In other words, for every scenario where HornetQ is less likely to be used it is more likely to be used in several others.

     

    As far as upgraders being the "biggest losers" in this regard, I think that's a two-way street.  For some it will be a loss, but for others it will be a gain.  At this point, it seems the winners outweigh the losers.

     

    If you are an EAP customer you can certainly raise this issue through that avenue.  Eventually your request will make its way to EAP product management.  As I'm sure you know, technical and philosophical considerations aren't always the most important in the complex landscape of software design.  Sometimes it just comes down to customer demand (for whatever underlying reason).

  • 29. Re: Does HornetQ support JBBC persistence
    mazz Master

    FWIW: I started a new thread that asks "how hard would it be for the community to implement this feature?" I'm hoping Clebert can chime in over there :-)

1 2 Previous Next