2 Replies Latest reply on Nov 14, 2012 6:11 AM by ryayon

    Agent doesn't send reports to server

    kbielecki

      Agent was running for a long time without problem. Then suddenly all resources discovered by agent went down.

      I've tried to restart agent, RHQ server, Jboss, install agent one more time, uninventory resources but reports are not send to server.

      Strange is that I have values in Platform Utilization but I can't connect to Jboss or Apache. Remaining agents are running without any problems.

       

      RHQ 4.3

      OS: Solaris 10

       

      2012-05-09 11:11:39,132 WARN  [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to synchronize local inventory with Server inventory for Resource [10129] and its descendants: null
      2012-05-09 11:11:39,132 WARN  [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Exception caught while running server discovery
      java.lang.RuntimeException: java.util.ConcurrentModificationException
          at org.rhq.core.pc.inventory.InventoryManager.synchInventory(InventoryManager.java:1072)
          at org.rhq.core.pc.inventory.InventoryManager.handleReport(InventoryManager.java:1006)
          at org.rhq.core.pc.inventory.AutoDiscoveryExecutor.call(AutoDiscoveryExecutor.java:121)
          at org.rhq.core.pc.inventory.AutoDiscoveryExecutor.run(AutoDiscoveryExecutor.java:92)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
          at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
          at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:619)
      Caused by: java.util.ConcurrentModificationException
          at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
          at java.util.HashMap$KeyIterator.next(HashMap.java:828)
          at org.rhq.core.pc.inventory.InventoryManager.mergeResource(InventoryManager.java:2708)
          at org.rhq.core.pc.inventory.InventoryManager.mergeUnknownResources(InventoryManager.java:2610)
          at org.rhq.core.pc.inventory.InventoryManager.synchInventory(InventoryManager.java:1041)
          ... 12 more
      2012-05-09 11:11:39,139 INFO  [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Found 1 servers.
      2012-05-09 11:11:39,139 INFO  [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Running runtime discovery scan rooted at [platform]
      2012-05-09 11:12:00,811 INFO  [ResourceDiscoveryComponent.invoker.daemon-2] (org.rhq.plugins.apache.ApacheServerDiscoveryComponent)- SNMPConf directive not found. Skipping SNMP configuration.
      2012-05-09 11:12:01,357 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Error in runtime discovery
      java.lang.NullPointerException
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:239)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:277)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:277)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResourceRecursive(RuntimeDiscoveryExecutor.java:158)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.runtimeDiscover(RuntimeDiscoveryExecutor.java:140)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.call(RuntimeDiscoveryExecutor.java:99)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.run(RuntimeDiscoveryExecutor.java:88)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
          at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
          at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:619)
      2012-05-09 11:12:01,368 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Error in runtime discovery
      java.lang.NullPointerException
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:239)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:277)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:277)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResourceRecursive(RuntimeDiscoveryExecutor.java:158)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.runtimeDiscover(RuntimeDiscoveryExecutor.java:140)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.call(RuntimeDiscoveryExecutor.java:99)
          at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.run(RuntimeDiscoveryExecutor.java:88)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
          at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
          at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:619)
      2012-05-09 11:12:01,378 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Error in runtime discovery
      
      
      or:
      
      
      2012-05-09 11:27:12,224 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Finished server sync for drift definitions. 0 added and 0 deleted in 30 ms
      2012-05-09 11:27:12,224 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Starting drift content sync...
      2012-05-09 11:27:12,224 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Finished drift content sync in 0 ms
      2012-05-09 11:27:12,292 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Starting server sync for drift definitions...
      2012-05-09 11:27:12,323 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Finished server sync for drift definitions. 0 added and 0 deleted in 31 ms
      2012-05-09 11:27:12,323 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Starting drift content sync...
      2012-05-09 11:27:12,323 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Finished drift content sync in 0 ms
      2012-05-09 11:27:12,688 INFO  [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Found 1 servers.
      2012-05-09 11:33:31,558 WARN  [Content.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Cannot get a resource container for an invalid resource ID=0
      2012-05-09 11:33:31,559 WARN  [Content.discovery-1] (rhq.core.pc.content.ContentDiscoveryRunner)- Exception received from component while attempting content retrieval
      org.rhq.core.clientapi.agent.PluginContainerException: Resource component container could not be retrieved for resource: 0
          at org.rhq.core.pc.util.ComponentUtil.getComponent(ComponentUtil.java:102)
          at org.rhq.core.pc.content.ContentManager.performContentDiscovery(ContentManager.java:418)
          at org.rhq.core.pc.content.ContentDiscoveryRunner.call(ContentDiscoveryRunner.java:123)
          at org.rhq.core.pc.content.ContentDiscoveryRunner.run(ContentDiscoveryRunner.java:90)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
          at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
          at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      
      
          at java.lang.Thread.run(Thread.java:619)
      2012-05-09 11:33:31,560 WARN  [Content.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Cannot get a resource container for an invalid resource ID=0
      

       

       

      I don't have any idea what I can check?