7 Replies Latest reply on Feb 1, 2010 4:39 AM by nfilotto

    How to evict a node when all its children have been evicted?

    nfilotto

      Hi All,

       

      Since the last past days, I have been working on finding a memory leak in my program. I finally identified it, in fact it is due to the fact that in JBC when all the sub nodes of a JBC node are evicted, the parent node is never evicted even if it has not been declared resident.

       

      Let's take a concrete example

      We assume that I have the following configuration

      {code}

        <eviction wakeUpInterval="5000">

            <default algorithmClass="org.jboss.cache.eviction.FIFOAlgorithm" eventQueueSize="1000000">

               <property name="maxNodes" value="1" />

            </default>

         </eviction>

      {code}

       

      if I first call cache.put(Fqn.fromElements("1","1"),key,value)

      If I then call cache.put(Fqn.fromElements("2"),key,value), the node "/1/1" is evicted but its parent node "/1" is not evicted

      If I then call cache.put(Fqn.fromElements("3"),key,value), the node "/2" is evicted but node "/1" is still present.

       

      This zombie node (i.e. node "/1") is the cause of my memory leak. So my question is, is there a way to make JBC evicts those kind of nodes?

       

      To limit the memory leak, I implemented a EvictionActionPolicy, but I would like to know if there is a better solution. If I don't have any other way, could you check my code below and tell me how I could improve it, because up to now, it fixes about 95% of the memory leak but it seems that it doesn't work 100% of the time:

      {code}

      public class MyEvictionActionPolicy implements EvictionActionPolicy

      {

         Cache<?, ?> cache;

         private static final Log log = LogFactory.getLog(DefaultEvictionActionPolicy.class);

       

         public void setCache(Cache<?, ?> cache)

         {

            this.cache = cache;

         }

       

         public boolean evict(Fqn fqn)

         {

            boolean result;

            try

            {

               if (log.isTraceEnabled()) log.trace("Evicting Fqn " + fqn);

               cache.evict(fqn);

               result = true;

            }

            catch (Exception e)

            {

               if (log.isDebugEnabled()) log.debug("Unable to evict " + fqn, e);

               result = false;

            }

            if (fqn.size() != 3)

            {

               return result;

            }

            try

            {

               Fqn parentFqn = fqn.getParent();

               if (parentFqn.get(0).equals(JBossCacheWorkspaceStorageCache.CHILD_NODES) || parentFqn.get(0).equals(

                     JBossCacheWorkspaceStorageCache.CHILD_PROPS))

               {

                  // The expected data structure is of type $CHILD_NODES/${node-id}/${sub-node-name} or

                  // $CHILD_PROPS/${node-id}/${sub-property-name}

                  Set<Object> names = cache.getChildrenNames(parentFqn);

                  if (names.isEmpty() || (names.size() == 1 && names.contains(fqn.get(2))))

                  {

                     if (log.isTraceEnabled()) log.trace("Evicting Fqn " + fqn);

                     cache.evict(parentFqn);

                  }

               }

            }

            catch (Exception e)

            {

               if (log.isDebugEnabled()) log.debug("Unable to evict " + fqn, e);

            }

            return result;

         }

      }

      {code}

       

      Thank you in advance for your help,

      BR,

      Nicolas Filotto

        • 1. Re: How to evict a node when all its children have been evicted?
          galder.zamarreno
          Nicolas, are you using FIFOAlgorithm just as an example here? Or do you actually use FIFOAlgorithm in your tests? I'm trying to figure out whether this could be an issue specific to FIFOAlgorithm or it could be affecting the more generally used LRUAlgorithm. Did you try LRUAlgorithm?
          • 2. Re: How to evict a node when all its children have been evicted?
            nfilotto
            No I'm using FIFO as an example, in our application we use LRU and we meet the same issue
            • 3. Re: How to evict a node when all its children have been evicted?
              galder.zamarreno
              I've created JBCACHE-1567
              • 4. Re: How to evict a node when all its children have been evicted?
                galder.zamarreno
                Nicolas, I've looked at this and I'm not sure this is a bug. In the scenario you mentioned /1/1 is not deleted as such but it's contents are emptied and that's the primary use case, since empty nodes shouldn't be taking that much space. Is the fact that empty nodes are left around is what's causing you issues? IOW, the key/value set to /1/1 should have been evicted in the test you mention.
                • 5. Re: How to evict a node when all its children have been evicted?
                  nfilotto
                  The node that is not evicted is "/1" (its child node "/1/1" is normally evicted) which is indeed an "empty node". I agree with you one node should not take so much memory but under heavy load this issue happens thousands of times in few minutes so this little memory leak becomes a huge one very quickly.
                  • 6. Re: How to evict a node when all its children have been evicted?
                    nzamosenchuk
                    Have some assumption, that proposed ActionPolicy may cause some problems, but not sure.
                    There is a piece of code in method evict()
                      // The expected data structure is of type $CHILD_NODES/${node-id}/${sub-node-name} or
                      // $CHILD_PROPS/${node-id}/${sub-property-name}
                    >>> Set<Object> names = cache.getChildrenNames(parentFqn); <<<
                      if (names.isEmpty() || (names.size() == 1 &&
                      names.contains(fqn.get(2))))
                    This code will be invoked if node, that is going to be evicted is from CHILD_NODES or CHILD_PROPS and it's fqn size is 3. (i.e.
                    $CHILD_PROPS/parentUUID1/childName1). In this case such code will be invoked:
                    cache.getChildrenNames(parentFqn);       (*)
                    Investigating forums and cache source code, I've found that this will trigger a VISIT event on JBC nodes, so all the nodes will be
                    refreshed, so they will be moved to the end of eviction queue.
                    Let's imagine situation, when we have lots of nodes in $CHILD_PROPS sub tree. We are not using them for a long time and some node like
                    $CHILD_PROPS/parentUUID15/childName100 is now going to be evicted. ParentNodeEvictionActionPolicy.evict() will invoke this code (*), so all nodes in $CHILD_PROPS/parentUUID1/* will be "visited", so they will not be evict. Aren't them? So there is a situation when lots of
                    really unused JBC nodes will always be visited (refreshed) and so will remain in cache.
                    Its just an idea and may be I'm mistaken.
                    • 7. Re: How to evict a node when all its children have been evicted?
                      nfilotto

                      Hi Nikolay,

                       

                      To avoid going through the interceptor chain, the line

                      >>> Set<Object> names = cache.getChildrenNames(parentFqn); <<<

                      could be replaced with

                      >>> Set<Object> names = ((CacheSPI)cache).getNode(parentFqn).getChildrenNamesDirect(); <<<