3 Replies Latest reply on Sep 12, 2011 9:56 AM by eadyos

    Hundreds of Timer threads in Jboss 4.2.2

    cindepalley

      We are using jboss 4.2.2 open source application server in production. we are seeing hundreds of timer threads (timer-1 to timer-100 etc) in the jvisualvm thread viewer...also the heap memory seems to be increasing day by day even though there is not much activity..

      one of the first observations is this timer thread numbers increasing faster and in wait mode but they are active even though we are not using this anywhere..

       

      I am sure this wont be a new issue but need help in resolving this quicker..

       

      After going the forums, this seems to be related to Jboss messaging but we are not using this anywhere and removed the jboss messaging file to from the deployment folder as part of performance tuning..

       

      any help is greatly appreciated...this is impacting our system..

       

      I am attaching the screen shots captured in a document..

       

      thanks in advance.

       

      best regards

      Yugandhar

        • 1. Hundreds of Timer threads in Jboss 4.2.2
          genman

          It's not 100% clear that the issue is an application issue or JBoss issue. Those threads don't really identify who created them, do they?

           

          What does your application do? Your doc links to

           

          https://issues.jboss.org/browse/JBREM-1005

           

          which is related to JBoss Remoting but not messaging. Are you using remote EJBs or something. And this might be alleviated by updating this component or going with AS 4.2.3 or higher.

          • 2. Hundreds of Timer threads in Jboss 4.2.2
            cindepalley

            ross

             

            thanks for the information..

             

            This is a J2ee web application which invokes remote EJBs from another application. This app went to production few months ago and we are going with a major release now..we started observing this pattern in the load testing and also with visualvm monitoring..

            Please find below the thread stack for couple of the timer threads..

            ----------

            "Timer-12" - Thread t@95

               java.lang.Thread.State: TIMED_WAITING

                at java.lang.Object.wait(Native Method)

                - waiting on <392cf455> (a java.util.TaskQueue)

                at java.util.TimerThread.mainLoop(Timer.java:509)

                at java.util.TimerThread.run(Timer.java:462)

             

               Locked ownable synchronizers:

                - None

             

            "Timer-2" - Thread t@67

               java.lang.Thread.State: WAITING

                at java.lang.Object.wait(Native Method)

                - waiting on <2fdd1bd6> (a java.util.TaskQueue)

                at java.lang.Object.wait(Object.java:485)

                at java.util.TimerThread.mainLoop(Timer.java:483)

                at java.util.TimerThread.run(Timer.java:462)

             

               Locked ownable synchronizers:

                - None

             

             

             

             

            -------------

             

            I did copy the jboss-remoting.jar file (JBossRemoting Version 2.2.2.SP9-preview) into the server/default/lib directory  from

            https://issues.jboss.org/browse/JBREM-1005

             

            I see another jar file jboss-remoting-int.jar in the same directory.

             

            but i still see the timer threads increasing continuously...not sure whether I am missing anything here..

             

            It will be highly useful to go with a an updated component instead of upgrading the server itself at this crutial time..we are planning to upgrade to 5.1 EAP in the next release..

             

            thanks very much in advance

             

            regards

            Yugandhar

            • 3. Re: Hundreds of Timer threads in Jboss 4.2.2
              eadyos

              I just wanted to add some information.

               

              I am having this same issue in Weblogic 10.3 under Jrockit

               

              The timer threads are slowly incrementing Timer-1 ... Timer-222

              They are locking on java.util.TaskQueue (1 instance per timer, not the same instance)

              Evetually the server crashes due to too many native threads being spawned.

               

              So I'm not sure if this is server related or application related.  In case it helps, here are some of the packages in use, it may help for others to cross-reference for determining what is spawning these things.

              Struts 2.0.11.1, Hessian RMI 3.1.3, Spring 2.0.5, ehcache 2.5.0, ibatis  2.2