-
15. Re: commit ordering issue
clebert.suconic Jul 13, 2010 12:24 PM (in response to clebert.suconic)I had written a test yesterday that would cause an out-of-order issue, and I have just committed it (PagingTest::disabled_testDepageDuringTransaction2)
The test will send a message before paging. sending 4 messages when the system enters into page. and another 5 with the system out of paging, and validating the order at the end. (it is failing with out of order now).
-
16. Re: commit ordering issue
jmesnil Jul 16, 2010 11:12 AM (in response to clebert.suconic)Clebert Suconic wrote:
I had written a test yesterday that would cause an out-of-order issue, and I have just committed it (PagingTest::disabled_testDepageDuringTransaction2)
The test will send a message before paging. sending 4 messages when the system enters into page. and another 5 with the system out of paging, and validating the order at the end. (it is failing with out of order now).
fiy, I re-applied your patch locally and it fixes the failing test you added PagingTest::disabled_testDepageDuringTransaction2.
I slightly modified the patch (i.e. making explicit that the paging uses a "copy" of the transaction without its TransactionOperations).
With this patch, messages which are routed after system went out of paging will be correctly routed with a transaction context.
I'm running the full test suite to check for regression and will commit the fix today (or on monday).
-
17. Re: commit ordering issue
timfox Jul 16, 2010 11:21 AM (in response to jmesnil)As mentioned before, I would like to see a proper explanation of the problem before any patches are applied.
-
18. Re: commit ordering issue
jmesnil Jul 19, 2010 4:10 AM (in response to ataylor)the issue is related to paging and transaction.
it occurs when the paging mode of a destination changes while messages are sent in a transaction
- we send a message in a tx to an adress
- we send multiple non-transacted messages to the address
=> the address starts to page
- we send other messages in the tx
=> messages are added to a PageOperation executed when the tx will commit
- we consume non-transacted messages from the queue
=> the address starts to depage
- we commit the tx
=> when the PageOperations are executed, the address is no longer paging and messages
will be routed by the post office *without* the transactional context in PageOperation.beforeCommit
=> these messages will be routed before the 1st message sent in the tx is routed (done in AddOperation.afterCommit)
=> out of order issue
with clebert's patch, we make sure that messages routed from PageOperations when the address is not paging will keep the transaction context so they remain in order.
-
19. Re: commit ordering issue
timfox Jul 19, 2010 5:31 AM (in response to jmesnil)Wouldn't a simpler fix be, if any parts of a transaction are paged, then all the transaction must be paged, irrespective of whether the address leaves paging?
-
20. Re: commit ordering issue
clebert.suconic Jul 19, 2010 9:45 AM (in response to timfox)When we Page on TX, we defer routing / paging at commit time. During commit time the message could leave page state, and we can't page at that point.
Even if we add method to page in blocks, or route in blocks (page everything or nothing, or route everything or nothing as you said), we could still have a case where the system won't be paging during commit and we would need to add messages back to the current transaction through routing.
i.e. Some way to change the current TX still needed.
-
21. Re: commit ordering issue
timfox Jul 19, 2010 9:52 AM (in response to clebert.suconic)Clebert Suconic wrote:
... and we would need to add messages back to the current transaction through routing.Why? Why not always page the messages even if the address is not paging any more. I.e. if at least one message in the tx is paged they all must be paged.
-
22. Re: commit ordering issue
clebert.suconic Jul 19, 2010 10:01 AM (in response to timfox)There's a caveat on paging... when the address is in page mode, everything must be paged up to the moment it is not paging any more.
Say, if we page when there are no more messages on the address. you could have a situation where nothing will be depaged. We don't have any threads watching for the sizes.. it all happens based on the size of the Address during ACK.
During the ACK we do a check:
if sizeOfQueue - maxSizeQueue < pageSize => depage
Having the possibility of paging when the address is empty would break depage.
-
23. Re: commit ordering issue
timfox Jul 19, 2010 10:07 AM (in response to clebert.suconic)Can't you just prompt a depage as soon as the tx commits?
-
24. Re: commit ordering issue
clebert.suconic Jul 19, 2010 10:19 AM (in response to timfox)Tim Fox wrote:
Can't you just prompt a depage as soon as the tx commits?
I would need to think about it. I have a bad feeling on races when having multiple producers and consumers.
-
25. Re: commit ordering issue
clebert.suconic Jul 19, 2010 3:38 PM (in response to clebert.suconic)The page system was made to be depaged during ACK.
It would be easy to have race conditions when having multiple producers and consumers doing it transactionally. I would need to spend some time on testing it. It wouldn't be an easier change at the end.