0 Replies Latest reply on Dec 9, 2009 6:55 AM by gaohoward

    Here is How I solve A Hang Problem

    gaohoward


      To make sure all ongoing calls on an object are finished before closing the object, an interceptor is used

      public class ClosedInterceptor
      {
      
       //synchronization is omitted here as irrelevant to the issue.
       public Object invoke(invocation)
       {
       if (method_is_close)
       {
       while (inUseCount > 0)
       {
       wait();
       }
       //now safely in close state.
       }
       else
       {
       ++inUseCount;
       invoke_method();
       --inUseCount;
       notifyAll();
       }
       }
      }
      


      You can see if the object's close() is being called by some thread, it will wait there until the inUserCount is zero.

      But a calling thread may hang in waiting if the following call sequence happens:

      1. The calling thread call a normal method on the object.
      2. The interceptor intercepts the call, increase inUseCount by one and go on to call the real method.
      3. During the mothod call, some failure happened and as part of the failure handling, relavent resources will be closed, including this object. So the object's close() is called.
      4. The interceptor intercepts the close() call, as the inUseCount has been increased by one, it will wait there.
      5. The result is that the thread is waiting for itself to return, which will never happen.

      To slove the problem, I think the key is to avoid the calling thread waiting on itself. For this purpose, I write a ThreadValveCounter util class. Using this class, the above code can be re-written as:

      public class ClosedInterceptor
      {
       ThreadValveCounter counter = new ThreadValveCounter();
      
       //synchronization is omitted here as irrelevant to the issue.
       public Object invoke(invocation)
       {
       if (method_is_close)
       {
       if (closed)
       {
       return;
       }
      
       counter.waitForCompletion();
       //now safely in close state.
       }
       else
       {
       counter.pushThread();
       object.invokeMethod(method);
       counter.popThread();
       }
       }
      }
      


      and Below is the code of ThreadValveCounter

      import java.util.Iterator;
      import java.util.concurrent.ConcurrentHashMap;
      
      /**
       * A ThreadValveCounter
       *
       * This is a utility class to avoid a calling thread waiting on itself. In cases where an object is accessed concurrently,
       * closing that object requires first waiting for all the current ongoing calls to be finished, which means all current calling
       * threads (except the one who is calling close()) returning from their respective method calls on that object. Sometimes
       * the calling thread who is calling close() can come from one of those who are calling normal methods. In that situation
       * this calling thread will wait forever as it is waiting itself to return.
       *
       * This class keeps a per-thread counter to avoid the above deadlock. Before waiting, the thread can use this class
       * to exempt itself from the counter, thus eliminating the possibility of hang on itself.
       *
       * @author <a href="mailto:hgao@redhat.com">Howard Gao</a>
       *
       * Created Dec 9, 2009 4:37:14 PM
       *
       *
       */
      public class ThreadValveCounter
      {
      
       private ConcurrentHashMap<Long, IntHolder> threadCounter = new ConcurrentHashMap<Long, IntHolder>();
      
       public void pushThread()
       {
       Long tid = Thread.currentThread().getId();
       IntHolder counter = threadCounter.get(tid);
       if (counter == null)
       {
       counter = new IntHolder();
       threadCounter.put(tid, counter);
       }
       counter.add();
       }
      
       public int popThread()
       {
       IntHolder counter = threadCounter.get(Thread.currentThread().getId());
       if (counter != null)
       {
       if (counter.release() == 0)
       {
       synchronized(threadCounter)
       {
       threadCounter.notifyAll();
       }
       }
       }
       return 0;
       }
      
       public void waitForCompletion()
       {
       exemptCurrentThread();
       synchronized (threadCounter)
       {
       while (getCallCount() > 0)
       {
       try
       {
       threadCounter.wait();
       }
       catch (InterruptedException e)
       {
       //ignore
       }
       }
       }
       }
      
       private void exemptCurrentThread()
       {
       IntHolder counter = threadCounter.get(Thread.currentThread().getId());
       if (counter != null)
       {
       counter.clear();
       }
       synchronized(threadCounter)
       {
       threadCounter.notifyAll();
       }
       }
      
       private int getCallCount()
       {
       synchronized(threadCounter)
       {
       Iterator<IntHolder> iter = threadCounter.values().iterator();
       int total = 0;
       while (iter.hasNext())
       {
       total += iter.next().get();
       }
       return total;
       }
       }
      
       private static class IntHolder
       {
       private int counter = 0;
      
       public void add()
       {
       counter++;
       }
      
       public int get()
       {
       return counter;
       }
      
       public int release()
       {
       counter--;
       return counter;
       }
      
       public void clear()
       {
       counter = 0;
       }
       }
      
      }