Hi Flavia,
Flavia Rainone wrote:
Hi!
I have a rule that refers to the local var $connHandlerFuture on its body:
RULE retrieve ConnectionHandlerFactory Future
CLASS org.jboss.remoting3.EndpointImpl
METHOD doConnect
HELPER org.jboss.remoting3.test.racecondition.FutureHelper
AT INVOKE org.jboss.remoting3.spi.ConnectionProvider.connect
IF TRUE
DO
debug("retrieving ConnectionHandlerFactory"),
trackFuture(org.jboss.remoting3.test.racecondition.FutureHelper.CONNECTION_HANDLER_FACTORY, $connHandlerFuture.getIoFuture())
ENDRULE
This rule is part of CancelOpeningChannelTestCase and, while the rule above works if I run the test isolated, the rule stops from working if I run all tests in a batch, with mvn test.
When you say run the test isolated what exactly do you mean? Are you using BMUnit to run it? Or are you using Byteman from the Java command line? I suspect that this relates to JIRA BYTEMAN-28 but I would like to know the details of the failure case to be sure I understand what is happening.
Flavia Rainone wrote:
After some investigation, I realized the failure is related with the previous test in the batch, CloseConnectingEndpointTestCase. Such test contains a rule that is applied to the same EndpointImpl.doConnect method:
RULE doConnect calls resourceUntick
CLASS org.jboss.remoting3.EndpointImpl
METHOD doConnect
AFTER INVOKE org.jboss.remoting3.EndpointImpl.resourceUntick
IF TRUE
DO
debug("ResourceUntick invoked by doConnect... waking closeAction"),
signalWake("resourceUntick for connection", true),
debug("ResourceUntick waiting for closeAction to complete before proceeding"),
# waitFor should expire if connect and closeAction are correctly synchronized, otherwise this test will hang
waitFor("connections to Array", 100)
ENDRULE
Debugging showed me that, somehow, on the first attempt to transform the bytecodes of EndpointImpl, for CloseConnectingEndpointTestCase, all local variables are accounted for (org.jboss.byteman.agent.adapter.RuleMethodAdapter.visitLocalVariable is invoked during the EndpointImpl bytecode visit process). But, when transformation for the problematic CancelOpeningChannelTestCase begins, no local variable is visited, and that's why my rule is not being applied to EndpointImpl.doConnect.
Could the retransformation process, performed to revert the transformation in the end of CloseConnectingEndpointTestCase execution, be erasing the local var info from the class? This is the impression I get while debugging, that the EndpointImpl.doConnect bytecodes are lacking the local var info when Byteman attempts to transform EndpointImpl for CancelOpeningChannelTestCase.
This may indeed be a possibility. When I last looked at the JVM code (i.e.when BYTEMAN-28 was reported) I thought the situation was as follows.
If the JVM loaded a class and a transformer modified it at load time then the JVM would retain the complete original bytecode. So, any subsequent attempt to retransform the class would start with the bytecode obtained from the class file and this would include the local variable information.
If the bytecode was not transformed at load time then the JVM would drop it. So, any later attempt to retransform the class required the JVM to reconstitute the bytecode from the internal in-memory represenation of the class and method code, meaning the bytecode would not include local variable info.
So, my impression was that if the bytecode was retained at load then it would be kept for every subsequent retransformation. However, from your evidence it sounds like it may throw away bytecode if some intermediate transform decided not to modify it. This may be a recent change or may be how it always was. In order to explain what I mean by that here is the scenario I am talking about in detail.
BMUNit executes the first test
The agent installs the rule for class EndpointImpl
Since EndpointImpl
has not yet been loaded it cannot be scheduled for retransformation
The test code runs and causes EndpointImpl
to be loaded
The agent transformer is passed the original loaded byecode for EndpointImpl
and modifies it
EndpointImpl
is transformed at load so the original bytecode is retained by the JVM
The test completes
The agent uninstalls the rule for class EndpointImpl
The agent schedules EndpointImpl
for retransformation
The agent is passed the original loaded byecode for EndpointImpl
and returns it unmodified
A) ?????
BMUNit executes the second test
The agent installs the 2nd rule for class EndpointImpl
Since EndpointImpl
has already been loaded it is scheduled for retransformation
B) The agent is passed ????? byecode for EndpointImpl
and modifies it
. . .
So, how do we fill in the blanks at A) and B) What I assumed on first reading the code was
A) The JVM retains the original bytecode
B) The agent is passed the original byecode for EndpointImpl
and modifies it
However, what might be happening is
A) The JVM drops the original loaded bytecode because it has not been transformed
B) The agent is passed the reconstituted byecode for EndpointImpl
and fails to modify it
I suspect this is what is happening but I'll have to check the JVM code to be sure.
Anyway, the upshot of this is that until BYTEMAN-28 is fixed you can only reliably use $localVar in your rules if you drive Byteman from the Java command line. Note that this is not the only way things can go wrong, merely a new wrinkle on the original problem. Imagine your first test did not use a rule attached to EndpointImpl
but instead that it ran some code which caused the class to be loaded. Your second test would still fail because the class would have been loaded and the original bytecode dropped.
I did implement a JVM patch for this a long time ago but for reasons far too complex to go into have not yet been able to get it into OpenJDK. I'm hoping I will be able to fix this in the next few months, assuming I don't get diverted from OpenJDK work yet again. It's a nice small patch which I can use as a way of learning how to jump all the hurdles whihc are in the way of pushing code upstream into OpenJDK.