I've just checked and indeed I confirm that it works perfectly without any workaround on 5.3 and 6.0. Unfortunately I have to use the version shipped into EAP 6.3 which is ISPN 5.2.x :-(

BTW what is the "primary owner" in case of a replicated cache? I understand the meaning in case of a distributed cache but it is not clear in case of a replicated cache

To reproduce, I launch 2 JVM with -Djava.net.preferIPv4Stack=true -Djgroups.bind_addr=127.0.0.1 as System properties, they both execute the next class:

import org.infinispan.AdvancedCache;
import org.infinispan.Cache;
import org.infinispan.configuration.cache.CacheMode;
import org.infinispan.configuration.cache.Configuration;
import org.infinispan.configuration.cache.ConfigurationBuilder;
import org.infinispan.configuration.global.GlobalConfigurationBuilder;
import org.infinispan.manager.DefaultCacheManager;
import org.infinispan.manager.EmbeddedCacheManager;
import org.infinispan.transaction.TransactionMode;
import org.infinispan.transaction.lookup.TransactionManagerLookup;

import java.util.Random;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;

import javax.transaction.TransactionManager;

public class PutNRemoveTest
{
  private static final int MAX_ITERATIONS = 20000;
  private static final int TOTAL_THREAD = 20;
  private static final String PARENT_ID = "$PARENT-ID";
  private static final String ID = Long.toString(new Random().nextLong());
  private static AtomicInteger COUNT = new AtomicInteger();

  private static void putNRemove(AtomicBoolean stop, AdvancedCache<String, String> cache) throws Exception
  {
      int iteration = COUNT.incrementAndGet();
      if (iteration >= MAX_ITERATIONS)
      {
        stop.compareAndSet(false, true);
        return;
      }
      final TransactionManager tm = cache.getAdvancedCache().getTransactionManager();
      String id = ID + "-" + System.currentTimeMillis() + "-" + iteration;
      try
      {
        tm.begin();
        cache.put(id, "foo");
        cache.remove(PARENT_ID);
        tm.commit();
        System.out.println("Entry '" + id + "' has been added");

        tm.begin();
        cache.remove(id);
        cache.remove(PARENT_ID);
        tm.commit();
        System.out.println("Entry '" + id + "' has been removed");
      }
      catch (Exception e)
      {
        System.err.println("Could not add/remove the entry '" + id + "' due to the error: " + e.getMessage());
        e.printStackTrace(System.err);
      }
  }

  public static void main(String[] args) throws Exception
  {
      GlobalConfigurationBuilder configBuilder = new GlobalConfigurationBuilder();
      configBuilder.transport().defaultTransport().addProperty("configurationFile", "udp.xml");
      EmbeddedCacheManager manager = new DefaultCacheManager(configBuilder.build());
      ConfigurationBuilder confBuilder = new ConfigurationBuilder();
      confBuilder.clustering().cacheMode(CacheMode.REPL_SYNC).stateTransfer().fetchInMemoryState(true);
      final TransactionManager tm = com.arjuna.ats.jta.TransactionManager.transactionManager();
      TransactionManagerLookup tml = new TransactionManagerLookup()
      {
        public TransactionManager getTransactionManager() throws Exception
        {
            return tm;
        }
      };
      confBuilder
        .transaction()
        .syncRollbackPhase(true)
        .transactionManagerLookup(tml).transactionMode(TransactionMode.TRANSACTIONAL);
    
      Configuration conf = confBuilder.build();
      manager.defineConfiguration("test", conf);
      Cache<String, String> c = manager.getCache("test");
      c.start();
      final AdvancedCache<String, String> cache = c.getAdvancedCache();
      final AtomicBoolean stop = new AtomicBoolean();
      final CountDownLatch startSignal = new CountDownLatch(1);
      final CountDownLatch endSignal = new CountDownLatch(TOTAL_THREAD);
      Runnable task = new Runnable()
      {
        public void run()
        {
            try
            {
              startSignal.await();
              while (!stop.get())
              {
                  putNRemove(stop, cache);
                  Thread.sleep(20);
              }
            }
            catch (Exception e)
            {
              System.err.println("Failed :" + e.getMessage());
            }
            finally
            {
              endSignal.countDown();
            }
        }
      };
      for (int i = 1; i <= TOTAL_THREAD; i++)
      {
        new Thread(task, "PutNRemoveTest-" + i).start();
      }
      System.out.println("First we wait 10 seconds to have enough time to launch other JVM");
      Thread.sleep(10000);
      System.out.println("Launching the threads with the ID=" + ID);
      startSignal.countDown();
      System.out.println("Waiting until we reach " + MAX_ITERATIONS + " iterations");
      endSignal.await();
      System.exit(0);
  }
}

You have 10 seconds to launch your JVM, then it will launch 20 threads that will do 20 K iterations in total, for each iteration it will put an entry and remove its "parent" then it will remove the entry and its "parent".

With ISPN 5.2.7.Final, you should get error of type "org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [10 seconds] on key [$PARENT-ID] for requestor [GlobalTransaction:<andromede-33120>:1717:local]! Lock held by [GlobalTransaction:<andromede-33120>:1714:local]"

Note 1: you will need arjuna (aka jbossts) 4.6.1-GA to run this test.

Note 2: this test works with ISPN 5.3 and 6.0 as already mentioned before

Hi Tristan,

Thank you for the info, but actually this is indirectly needed for http://www.jboss.org/gatein which is a supported project so we need to use the versions of the third party librairies shipped into EAP.

Nicolas

Good point! Increasing the pool size allowed me to avoid the TimeoutException without any workaround.

Thank you Pedro, this solved my issue, it is strange that we don't face the issue with the default pool configuration with ISPN 5.3 and 6.0 but it is another story

I see, indeed it makes perfect sense, thank you for this precious info