XA_RBTIMEOUT is indeed preferable, but is handled by JBossTS in the same manner as XAER_NOTA. The distinction in spec terms is that XAER_NOTA is an error rather than an information code, which is not quite right in this case, since the timeout is actually working as intended. However, implementation considerations in the RM should probably trump such spec minutia.

Timeouts are useful where the RM uses locking and allowing those locks to be held too long causes throughput problems. If the RM is non-blocking then it becomes a question of resource allocation. If holding the tx open requires no resources and holds no locks, timeouts are irrelevant. Otherwise you need them for gc, or you need very robust connection handling so that you can do cleanup of unprepared tx on connection drop. TCP/IP being what it is, you won't always detect such drops unless you happen to have a heartbeat protocol between client and server.

Jeff, in addition to what Jonathan said, you shouldn't rely on the TM for timeouts, particularly if your RM and TM can fail independently. If the TM fails before prepare, say, then it won't know anything about the RM (no log) and so won't ever try to timeout. In that case your RM needs to do it.

Plus let's not forget: if your RM times out after agreeing to prepare then you need to return a heuristic outcome (and remember it durably until forget is called).