I have a question concerning HornetQ running in JBoss 6.0.0.
I have a scenario whereby I want to track a series of messages posted to hornetq in an "ImportLog" entity I have created. To give a little further detail, the messages posted to HornetQ contain records that are to be imported into the database. For performance reasons, we like to limit the batch of records inserted at any one time. If our limit is 500 and a user decides to import 2000, we would create 4 messages for example.
The ImportLog is a simple entity that helps us track and record stats of the import as a whole. So each message that processes will load the ImportLog entity and potentially update. The import queue has the maxSessions property set to 1, which means that the messages correctly process one-at-a-time.
Before any messages are posted to HornetQ, the ImportLog entity is created. The id of the ImportLog entity is then passed as part of the object message.
Radomly, everything will work perfectly fine. However, very frustratingly, the first message picked up will often claim that it cannot find the ImportLog entity (i.e. ImportLog log = em.find(ImportLog, id)). It appears that the message is being consumed before Hibernate has committed the import log to the database? Interestingly subsequent messages will find the ImportLog perfectly.
I have tried various things to "force" hibernate to commit the entity so that the first message will always find the entity in the database. This includes calling flush(), manually committing the transaction after the ImportLog entity is initially created, along with other tricks I have read about that include calling session.evict() and session.update().
Has anybody else found this problem? Is there a way to overcome or am I missing something / abusing functionality here?
Any thoughts would be greatly appreciated
@Clebert, i disagree that this case is "just some misuse on Hibernate".
The link provided by Leos descibes exactly the problem I am facing. In the short term, I am having to rewrite code and lose minor functionality to overcome. In the long term, I am investigating the solutions put forward in the link.
As I have described in that Jira issue JMS_SPEC-28, if everybody (HornetQ and Hibernate in this case) just states "not my issue", it is not going to work. In this case it would help if HornetQ would be able to implement the marker interface com.arjuna.ats.jta.resources.EndXAResource for its XAResource. I understand that creating such direct dependency on Arjuna's API is not acceptable, but providing some extensibility (API for providing an optional wrapper for XAResource maybe?) could be possible, couldn't it?
To be fair, this really isn't a HornetQ issue. Even Nigel said, "...this doesn't sound like something that can be tackled at the level of an individual resource manager such as JMS." The problem should be addressed by the JTA specification.
As you pointed out, JBossTS (the transaction manager used by JBoss AS) has an extension that will allow more control of this, but there are handful of problems with having our XAResource implementation also implement com.arjuna.ats.jta.resources.EndXAResource. These are the issues I can think of off the top of my head:
- We'd need to provide 3 implementations - one that implemented com.arjuna.ats.jta.resources.EndXAResource, one that implemented com.arjuna.ats.jta.resources.StartXAResource, and one that didn't implement either (which would be default). We wouldn't want to change the default behavior or only provide the option to make it the last resource committed.
- This would only help when JBossTS was used as the TM. Since HornetQ can be deployed embedded in other containers and accessed remotely from other containers even when run within JBoss AS (or standalone) this is far from a complete solution.
- This is essentially a hack which likely brings unwanted dependencies.
As I understand it, this is a known weakness in JTA (which is why the JBossTS work-around exists). However, I'm not sure what measures have been undertaken to resolve it - if any. I'm interested to see what happens on http://java.net/jira/browse/JAVAEE_SPEC-1.
I'll ask Clebert what he wants to do here as far as it concerns HornetQ as it's not clear to me if this is really something we want to do.
also read http://jbossts.blogspot.co.uk/2011/04/messagingdatabase-race-conditions.html which has some good insight into it. It also suggest a way around the issue altho slightly clunky.
As far as I am aware this is being tackled in the JTA 1.2 spec.
maybe one thing you can do is use a user managed tx and enlist the resources in the order you want them commited.