I am currently checking that HornetQ XAResource behave correctly wrt to transaction timeout. There is a whole discussion about it (http://www.jboss.org/index.html?module=bb&op=viewtopic&t=144955) but I have some questions which are more general about how the TM and RM must cooperate.
If the RM accepts the transaction timeout set by the TM (XAResource.setTransactionTimeout() returns true) and rolls back a tx after the timeout, is it correct to return XA_ERRNOTA when the TM calls prepare() on the RM? Or MUST we return XA_RBTIMEOUT?
AIUI, returning XA_RBTIMEOUT gives the TM more contextual info but it will be handle the same way than XA_ERRNOTA. Is it the case for JBoss TS?
If we can return XAERR_NOTA, this means the RM can forget about the XID as soon as the tx timeout is hit. If we must return XA_RBTIMEOUT, it must keep the XID until the TM calls prepare().
Another question: Is there a real advantage for the RM to handle timeout?
The more I look at this issue for HornetQ, the less I am convinced we should handle tx timeout at all. It seems simpler to let the TM drive the whole thing.
The only issue I see with the RM not dealing with tx timeout is when the RM and TM runs in separate processes and the TM crashes.
XA_RBTIMEOUT is indeed preferable, but is handled by JBossTS in the same manner as XAER_NOTA. The distinction in spec terms is that XAER_NOTA is an error rather than an information code, which is not quite right in this case, since the timeout is actually working as intended. However, implementation considerations in the RM should probably trump such spec minutia.
Timeouts are useful where the RM uses locking and allowing those locks to be held too long causes throughput problems. If the RM is non-blocking then it becomes a question of resource allocation. If holding the tx open requires no resources and holds no locks, timeouts are irrelevant. Otherwise you need them for gc, or you need very robust connection handling so that you can do cleanup of unprepared tx on connection drop. TCP/IP being what it is, you won't always detect such drops unless you happen to have a heartbeat protocol between client and server.
Jeff, in addition to what Jonathan said, you shouldn't rely on the TM for timeouts, particularly if your RM and TM can fail independently. If the TM fails before prepare, say, then it won't know anything about the RM (no log) and so won't ever try to timeout. In that case your RM needs to do it.
Plus let's not forget: if your RM times out after agreeing to prepare then you need to return a heuristic outcome (and remember it durably until forget is called).