8 Replies Latest reply on Oct 18, 2011 2:14 AM by jfclere

    String index out of range: -1

    yves.p

      Hello everyone

       

      I have a Problem with mod_cluster Version 1.1.1 an up. When an Applicationserver trys to register on mod_cluster the following error gets thrown. I added a log statment in DefaultMCMPHandler that prints out the response of the Apache (in bold):

      2011-09-20 09:10:34,759 INFO  [org.jboss.modcluster.ModClusterService] (main) Initializing mod_cluster 1.1.3.Final

      2011-09-20 09:10:34,855 DEBUG [org.jboss.modcluster.ModClusterService] (main) Received server start event

      2011-09-20 09:10:45,107 DEBUG [org.jboss.modcluster.ModClusterService] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Check status for engine [jboss.web]

      2011-09-20 09:10:45,108 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Sending command [org.jboss.modcluster.mcmp.impl.DefaultMCMPRequest{requestType=INFO,wildcard=false,jvmRoute=null,parameters={}}] to proxy [jsplb01x.umobi.mobicorp.test/10.32.34.139:80]

      2011-09-20 09:10:45,111 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Parsing responseStatus:HTTP/1.1 200 OK

      2011-09-20 09:10:45,133 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) [org.jboss.modcluster.mcmp.impl.DefaultMCMPRequest{requestType=CONFIG,wildcard=false,jvmRoute=defaultmetro_u_01,parameters={Balancer=defaultmetro, Host=10.32.34.70, Maxattempts=1, Port=8009, Type=ajp}}, org.jboss.modcluster.mcmp.impl.DefaultMCMPRequest{requestType=ENABLE-APP,wildcard=false,jvmRoute=defaultmetro_u_01,parameters={Alias=localhost, Context=/blublu/sampleApp}}]

      2011-09-20 09:10:45,134 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Sending command [org.jboss.modcluster.mcmp.impl.DefaultMCMPRequest{requestType=CONFIG,wildcard=false,jvmRoute=defaultmetro_u_01,parameters={Balancer=defaultmetro, Host=10.32.34.70, Maxattempts=1, Port=8009, Type=ajp}}] to proxy [jsplb01x.umobi.mobicorp.test/10.32.34.139:80]

      2011-09-20 09:10:45,134 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Parsing responseStatus:1f53

      2011-09-20 09:10:45,134 INFO  [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Error parsing response header for command CONFIG

      java.lang.StringIndexOutOfBoundsException: String index out of range: -1

          at java.lang.String.substring(String.java:1937)

          at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.sendRequest(DefaultMCMPHandler.java:667)

          at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.sendRequests(DefaultMCMPHandler.java:476)

          at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.status(DefaultMCMPHandler.java:417)

          at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.status(DefaultMCMPHandler.java:377)

          at org.jboss.modcluster.ModClusterService.status(ModClusterService.java:480)

          at org.jboss.modcluster.catalina.CatalinaEventHandlerAdapter.lifecycleEvent(CatalinaEventHandlerAdapter.java:323)

          at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)

          at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1348)

          at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1612)

          at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1601)

          at java.lang.Thread.run(Thread.java:619)

      2011-09-20 09:10:45,135 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Error [null: null: {4}] sending command CONFIG to proxy jsplb01x.umobi.mobicorp.test/10.32.34.139:80, configuration will be reset

      2011-09-20 09:10:55,523 DEBUG [org.jboss.modcluster.ModClusterService] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Check status for engine [jboss.web]

       

      In the httpd debug logs I found the following:

      [Mon Sep 19 18:14:48 2011] [debug] mod_manager.c(2296): manager_handler STATUS (/) processing: "JVMRoute=orpaws_x_01&Load=89"

      [Mon Sep 19 18:14:48 2011] [debug] mod_manager.c(1394): Processing STATUS

      [Mon Sep 19 18:14:48 2011] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (10.32.34.239)

      [Mon Sep 19 18:14:48 2011] [debug] proxy_util.c(2067): proxy: connecting ajp://10.32.34.239:8009/ to 10.32.34.239:8009

      [Mon Sep 19 18:14:48 2011] [debug] proxy_util.c(2193): proxy: connected ajp://10.32.34.239:8009/ to 10.32.34.239:8009

      [Mon Sep 19 18:14:48 2011] [debug] mod_proxy_cluster.c(716): ajp_cping_cpong: Done

      [Mon Sep 19 18:14:48 2011] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (10.32.34.239)

      [Mon Sep 19 18:14:48 2011] [debug] mod_manager.c(2339): manager_handler STATUS  OK

      [Mon Sep 19 18:14:48 2011] [debug] mod_headers.c(743): headers: ap_headers_output_filter()

      [Mon Sep 19 18:14:48 2011] [debug] mod_manager.c(1653): manager_trans INFO (/)

      [Mon Sep 19 18:14:48 2011] [debug] mod_manager.c(2296): manager_handler INFO (/) processing: ""

      [Mon Sep 19 18:14:48 2011] [debug] mod_headers.c(743): headers: ap_headers_output_filter()

      [Mon Sep 19 18:14:48 2011] [debug] mod_manager.c(2339): manager_handler INFO  OK

      [Mon Sep 19 18:14:48 2011] [debug] mod_manager.c(1653): manager_trans CONFIG (/)

      [Mon Sep 19 18:14:48 2011] [debug] mod_manager.c(2296): manager_handler CONFIG (/) processing: "JVMRoute=defaultmetro_u_01&Balancer=defaultmetro&Host=10.32.34.70&Maxattempts=1&Port=8009&Type=ajp"

      [Mon Sep 19 18:14:48 2011] [debug] mod_manager.c(2339): manager_handler CONFIG  OK

      [Mon Sep 19 18:14:48 2011] [debug] mod_headers.c(743): headers: ap_headers_output_filter()

      [Mon Sep 19 18:14:48 2011] [info] [client 10.32.34.116] (32)Broken pipe: core_output_filter: writing data to the network

      [Mon Sep 19 18:14:49 2011] [debug] ssl_engine_io.c(1882): OpenSSL: read 5/5 bytes from BIO#94f3480 [mem: 954c028] (BIO dump follows)

       

      When I restart the Apache the JBoss registert ok, but when I restart the the JBoss the registration doesn't work. The registration seams to work after a several retries from the JBoss (takes more than an houre). Also the registration works if there are not as many nodes registered (there are 26 JBoss Nodes and 42 Contexts).

       

      When I exchange the mod_cluster library in the mod_cluster.sar with the 1.1.0 Version everything works fine

      011-09-20 11:24:55,205 INFO  [org.jboss.modcluster.ModClusterService] (main) Initializing mod_cluster 1.1.0.Final

      2011-09-20 11:24:55,230 DEBUG [org.jboss.modcluster.ModClusterService] (main) Received server start event

      2011-09-20 11:25:05,763 DEBUG [org.jboss.modcluster.ModClusterService] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Check status for engine [jboss.web]

      2011-09-20 11:25:05,764 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Sending command [org.jboss.modcluster.mcmp.impl.DefaultMCMPRequest{requestType=INFO,wildcard=false,jvmRoute=null,parameters={}}] to proxy [jsplb01x.umobi.mobicorp.test/10.32.34.139:80]

      2011-09-20 11:25:05,790 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPResponseParser] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Node: [1],Name: tariffing_def_x_01,Balancer: tariffing_def,LBGroup: ,Host: 10.32.34.58,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 91

      Node: [2],Name: taskclient_x_01,Balancer: taskclient,LBGroup: ,Host: 10.32.34.173,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 18,Read: 67784,Transfered: 23515,Connected: 0,Load: 91

      Node: [3],Name: bonusmalus_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.152,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 92

      Node: [4],Name: cmqtis_x_01,Balancer: cmqtis,LBGroup: ,Host: 10.32.35.173,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 92

      Node: [5],Name: contract_v3_x_01,Balancer: contract_v3,LBGroup: ,Host: 10.32.35.145,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 75,Read: 1351895,Transfered: 257781,Connected: 1,Load: 90

      Node: [6],Name: isdv_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.159,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 99,Read: 2337010,Transfered: 29617,Connected: 0,Load: 91

      Node: [7],Name: defaultmetro_u_01,Balancer: defaultmetro,LBGroup: ,Host: 10.32.34.70,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: -1

      Node: [8],Name: beleg_x_01,Balancer: beleg,LBGroup: ,Host: 10.32.35.139,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 92

      Node: [9],Name: applicationlocking_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.35.71,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 12,Read: 92822,Transfered: 7035,Connected: 0,Load: 92

      Node: [10],Name: ecm_import_x_01,Balancer: ecm_import,LBGroup: ,Host: 10.32.34.199,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 4,Read: 1852,Transfered: 1824,Connected: 0,Load: 86

      Node: [11],Name: tariffing_op_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.196,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 163,Read: 2537139,Transfered: 4185006,Connected: 0,Load: 92

      Node: [12],Name: orpaws_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.239,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 16,Read: 55624,Transfered: 12477,Connected: 0,Load: 90

      Node: [13],Name: contract_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.26,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 2,Read: 72873,Transfered: 1539,Connected: 0,Load: 89

      Node: [14],Name: odsservice_x_01,Balancer: odsservice,LBGroup: ,Host: 10.32.34.81,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 5,Read: 27141,Transfered: 1725,Connected: 0,Load: 91

      Node: [15],Name: ecm_retrieval_x_01,Balancer: ecm_retrieval,LBGroup: ,Host: 10.32.34.200,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 144,Read: 718194,Transfered: 18266,Connected: 0,Load: 88

      Node: [16],Name: defa_x_01,Balancer: defa,LBGroup: ,Host: 10.32.35.110,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 676,Read: 14832365,Transfered: 29650,Connected: 1,Load: 74

      Node: [17],Name: rdi_x_01,Balancer: rdi,LBGroup: ,Host: 10.32.35.170,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 92

      Node: [18],Name: processinfo_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.179,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 92

      Node: [19],Name: pov_x_01,Balancer: pov,LBGroup: ,Host: 10.32.34.136,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 119,Read: 3234640,Transfered: 2270334,Connected: 0,Load: 90

      Node: [20],Name: contract_ac_x_01,Balancer: contract_ac,LBGroup: ,Host: 10.32.34.112,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 5,Read: 2140,Transfered: 2100,Connected: 0,Load: 91

      Node: [21],Name: moscontroller_x_01,Balancer: moscontroller,LBGroup: ,Host: 10.32.35.177,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 3,Read: 1011,Transfered: 723,Connected: 0,Load: 90

      Node: [22],Name: pas_interface_v2_x_01,Balancer: pas_interface_v2,LBGroup: ,Host: 10.32.35.144,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 11,Read: 9763,Transfered: 8653,Connected: 0,Load: 92

      Node: [23],Name: mmp_x_01,Balancer: mmp,LBGroup: ,Host: 10.32.35.47,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 4,Read: 2183,Transfered: 0,Connected: 0,Load: 90

      Node: [24],Name: ub_x_01,Balancer: ub,LBGroup: ,Host: 10.32.34.128,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 59,Read: 514040,Transfered: 3182,Connected: 0,Load: 88

      Node: [25],Name: avos_defa_x_01,Balancer: avos_defa,LBGroup: ,Host: 10.32.34.79,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 8,Read: 4149,Transfered: 3831,Connected: 0,Load: 88

      Node: [26],Name: processinfo_v2_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.35.74,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 67,Read: 75683,Transfered: 45347,Connected: 0,Load: 91

      Vhost: [1:1:1], Alias: localhost

      Vhost: [2:1:2], Alias: localhost

      Vhost: [3:1:3], Alias: localhost

      Vhost: [4:1:4], Alias: localhost

      Vhost: [5:1:5], Alias: localhost

      Vhost: [6:1:6], Alias: localhost

      Vhost: [8:1:8], Alias: localhost

      Vhost: [9:1:9], Alias: localhost

      Vhost: [10:1:10], Alias: localhost

      Vhost: [11:1:11], Alias: localhost

      Vhost: [12:1:12], Alias: localhost

      Vhost: [13:1:13], Alias: localhost

      Vhost: [14:1:14], Alias: localhost

      Vhost: [15:1:15], Alias: localhost

      Vhost: [16:1:16], Alias: localhost

      Vhost: [17:1:17], Alias: localhost

      Vhost: [18:1:18], Alias: localhost

      Vhost: [19:1:19], Alias: localhost

      Vhost: [20:1:20], Alias: localhost

      Vhost: [21:1:21], Alias: localhost

      Vhost: [22:1:22], Alias: localhost

      Vhost: [23:1:23], Alias: localhost

      Vhost: [24:1:24], Alias: localhost

      Vhost: [25:1:25], Alias: localhost

      Vhost: [26:1:26], Alias: localhost

      Context: [1:1:1], Context: /ts_webservice_def_x_v1, Status: ENABLED

      Context: [2:1:2], Context: /taskclient_webservice-1.0, Status: ENABLED

      Context: [3:1:3], Context: /objectpartner_webservice_x, Status: ENABLED

      Context: [3:1:4], Context: /bonusmalus_webservice_x, Status: ENABLED

      Context: [4:1:5], Context: /claimdata_webservice-1.0, Status: ENABLED

      Context: [5:1:6], Context: /contract_defa_webservice_v3_x, Status: ENABLED

      Context: [5:1:7], Context: /contract_webservice_v3_x, Status: ENABLED

      Context: [6:1:8], Context: /isdv_webservice2_x, Status: ENABLED

      Context: [8:1:10], Context: /beleg_webservice_v1_x, Status: ENABLED

      Context: [9:1:11], Context: /applicationlocking_webservice_x, Status: ENABLED

      Context: [10:1:12], Context: /dms.importer.admintool_x, Status: ENABLED

      Context: [10:1:13], Context: /dms.importer.delivery_webservice_x, Status: ENABLED

      Context: [11:1:14], Context: /bwb_webservice_x_v1, Status: ENABLED

      Context: [11:1:15], Context: /ts_webservice_op_x_v1, Status: ENABLED

      Context: [12:1:16], Context: /orpa_shortinfo_ws_v2, Status: ENABLED

      Context: [12:1:17], Context: /orpa_details_ws_v1_1, Status: ENABLED

      Context: [13:1:18], Context: /productstructureservice_webservice_x, Status: ENABLED

      Context: [13:1:19], Context: /contract_webservice_x, Status: ENABLED

      Context: [13:1:20], Context: /contract_defa_webservice_x, Status: ENABLED

      Context: [14:1:21], Context: /odsservice_v1, Status: ENABLED

      Context: [15:1:22], Context: /dms.dmsviewer_x, Status: ENABLED

      Context: [15:1:23], Context: /dms.retrieval.contentpresentation_x, Status: ENABLED

      Context: [15:1:24], Context: /dms.doctypeinfo.service_webservice_x, Status: ENABLED

      Context: [15:1:25], Context: /dms.retrieval_webservice_x, Status: ENABLED

      Context: [16:1:26], Context: /taskclient_x, Status: ENABLED

      Context: [16:1:27], Context: /defaservices_webservice_x, Status: ENABLED

      Context: [16:1:28], Context: /defaservices_x, Status: ENABLED

      Context: [17:1:29], Context: /rdi_webservice_v1_x, Status: ENABLED

      Context: [18:1:30], Context: /processinfo_webservice_x, Status: ENABLED

      Context: [19:1:31], Context: /povws_contract_v2_x, Status: ENABLED

      Context: [19:1:32], Context: /povws_product_v3_x, Status: ENABLED

      Context: [19:1:33], Context: /povws_product_v2_x, Status: ENABLED

      Context: [19:1:34], Context: /povws_print_v1_x, Status: ENABLED

      Context: [20:1:35], Context: /sanacontractaccesscontrolservice_ws_x, Status: ENABLED

      Context: [21:1:36], Context: /mos_webservice-0.2, Status: ENABLED

      Context: [22:1:37], Context: /pasinterface_webservice_v2_x, Status: ENABLED

      Context: [23:1:38], Context: /mmp, Status: ENABLED

      Context: [23:1:39], Context: /mmp_admin, Status: ENABLED

      Context: [24:1:40], Context: /ub_webgui, Status: ENABLED

      Context: [24:1:41], Context: /ub_webservice_v2_x, Status: ENABLED

      Context: [25:1:42], Context: /activebpel-cert, Status: ENABLED

      Context: [26:1:43], Context: /processinfo_webservice_v2_x, Status: ENABLED

       

      2011-09-20 11:25:05,817 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPResponseParser] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) {processinfo_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@1c68b47], tariffing_op_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@5b151a], rdi_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@1e20514], moscontroller_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@1bb7466], odsservice_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@1d6ed05], mmp_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@16301f4], avos_defa_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@1154838], ub_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@385516], defa_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@5a25d3], pov_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@695b85], bonusmalus_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@f7a61b], processinfo_v2_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@ba6e80], tariffing_def_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@1acd265], isdv_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@1290aac], taskclient_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@17bdc3e], contract_ac_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@b889fb], contract_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@16ac549], cmqtis_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@be77d8], pas_interface_v2_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@ae20f5], beleg_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@1f5d80d], defaultmetro_u_01=[], orpaws_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@13b3f83], ecm_import_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@c6b216], contract_v3_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@d5efe2], ecm_retrieval_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@1780e0b], applicationlocking_x_01=[org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$VirtualHostImpl@e2262b]}

      2011-09-20 11:25:05,832 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) [org.jboss.modcluster.mcmp.impl.DefaultMCMPRequest{requestType=CONFIG,wildcard=false,jvmRoute=defaultmetro_u_01,parameters={Balancer=defaultmetro, Host=10.32.34.70, Maxattempts=1, Port=8009, Type=ajp}}, org.jboss.modcluster.mcmp.impl.DefaultMCMPRequest{requestType=ENABLE-APP,wildcard=false,jvmRoute=defaultmetro_u_01,parameters={Alias=localhost, Context=/blublu/sampleApp}}]

      2011-09-20 11:25:05,832 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Sending command [org.jboss.modcluster.mcmp.impl.DefaultMCMPRequest{requestType=CONFIG,wildcard=false,jvmRoute=defaultmetro_u_01,parameters={Balancer=defaultmetro, Host=10.32.34.70, Maxattempts=1, Port=8009, Type=ajp}}] to proxy [jsplb01x.umobi.mobicorp.test/10.32.34.139:80]

      2011-09-20 11:25:05,834 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Sending command [org.jboss.modcluster.mcmp.impl.DefaultMCMPRequest{requestType=ENABLE-APP,wildcard=false,jvmRoute=defaultmetro_u_01,parameters={Alias=localhost, Context=/blublu/sampleApp}}] to proxy [jsplb01x.umobi.mobicorp.test/10.32.34.139:80]

      2011-09-20 11:25:05,876 TRACE [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Sending command [org.jboss.modcluster.mcmp.impl.DefaultMCMPRequest{requestType=STATUS,wildcard=false,jvmRoute=defaultmetro_u_01,parameters={Load=90}}] to proxy [jsplb01x.umobi.mobicorp.test/10.32.34.139:80]

      2011-09-20 11:25:16,356 DEBUG [org.jboss.modcluster.ModClusterService] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Check status for engine [jboss.web]

       

      Spec:

      Centos Linux

      JBoss 5.1

      mod_cluster-1.1.3.Final-bin

      mod_cluster-1.1.3.Final-linux2-x86-ssl

       

      I attached the httpd and JBoss configuration.

       

      Any clues?

       

      Update: added the tcpdump from the startup of JBoss with mod_cluster 1.1.0 and 1.1.3. The response from httpd seams to be ok but contains some extra characters, parsing error?

       

      Message was edited by: Yves Peter

        • 1. Re: String index out of range: -1
          rhusar

          Hey Yves,

           

          after just a very brief look the problem seems to be

           

          Broken pipe: core_output_filter: writing data to the network

           

          and that causes the other probems. But that could be anything, simply one end closes before sending response is complete. Maybe misconfiguration, network problem, fiirewall, etc. Can you try to reproduce that on different machine?

           

          Rado

          • 2. Re: String index out of range: -1
            yves.p

            Hey Rado

             

            It happens on a different machine too, also it works correctly with mod_cluster 1.1.0.

            If you look at the tcpdum output (with) you can see, that the httpd server is able to write back to the JBoss. JBoss throws the exception (String index out of range: -1) and closes the pipe before reading all the content which shows up as "Broken pipe: core_output_filter: writing data to the network" in the httpd logs, I think...

             

            Here the tcp stream form JBoss 1.1.0 vs. 1.1.3:

             

            1.1.0:

            INFO / HTTP/1.0
            Content-Length: 0
            User-Agent: ClusterListener/1.0
            Connection: Keep-Alive
            
            HTTP/1.1 200 OK
            Date: Tue, 20 Sep 2011 11:17:36 GMT
            Server: Apache
            Vary: Accept-Encoding
            Connection: close
            Content-Type: text/plain
            Node: [1],Name: tariffing_def_x_01,Balancer: tariffing_def,LBGroup: ,Host: 10.32.34.58,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 91
            Node: [2],Name: taskclient_x_01,Balancer: taskclient,LBGroup: ,Host: 10.32.34.173,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 113,Read: 1555249,Transfered: 153954,Connected: 0,Load: 92
            Node: [3],Name: bonusmalus_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.152,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 91
            Node: [4],Name: cmqtis_x_01,Balancer: cmqtis,LBGroup: ,Host: 10.32.35.173,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,nected: 0,Load: 91
            Node: [8],Name: beleg_x_01,Balancer: beleg,LBGroup: ,Host: 10.32.35.139,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 91
            Node: [9],Name: applicationlocking_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.35.71,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 66,Read: 139993,Transfered: 46871,Connected: 0,Load: 92
            Node: [10],Name: ecm_import_x_01,Balancer: ecm_import,LBGroup: ,Host: 10.32.34.199,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 4,Read: 1852,Transfered: 1824,Connected: 0,Load: 86
            Node: [11],Name: tariffing_op_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.196,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 287,Read: 8337915,Transfered: 11243610,Connected: 0,Load: 89
            Node: [12],Name: orpaws_x_01,Balancer: myclus8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 11,Read: 30681,Transfered: 5403,Connected: 0,Load: 91
            Node: [15],Name: ecm_retrieval_x_01,Balancer: ecm_retrieval,LBGroup: ,Host: 10.32.34.200,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 505,Read: 2295181,Transfered: 106013,Connected: 0,Load: 88
            Node: [16],Name: defa_x_01,Balancer: defa,LBGroup: ,Host: 10.32.35.110,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 2463,Read: 76471833,Transfered: 152332,Connected: 0,Load: 72
            Node: [17],Name: rdi_x_01,Balancer: rdi,LBGroup: ,Host: 10.32.35.170,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 92
            Node: [18],Name: processinfo_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.179,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Ead: 92
            Node: [21],Name: moscontroller_x_01,Balancer: moscontroller,LBGroup: ,Host: 10.32.35.177,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 3,Read: 1011,Transfered: 723,Connected: 0,Load: 90
            Node: [22],Name: pas_interface_v2_x_01,Balancer: pas_interface_v2,LBGroup: ,Host: 10.32.35.144,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 61,Read: 63427,Transfered: 47789,Connected: 0,Load: 91
            Node: [23],Name: mmp_x_01,Balancer: mmp,LBGroup: ,Host: 10.32.35.47,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 76,Read: 873613,Transfered: 285,Connected: 0,Load: 91
            Node: [24],Name: ub_x_01,Balancer: ub,LBGroup: ,Host: 10.32.34.128,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 1713,Read: 17212098,Transfered: 40676,Connected: 0,Load: 79
            Node: [25],Name: avos_defa_x_01,Balancer: avos_defa,LBGroup:st
            Vhost: [4:1:4], Alias: localhost
            Vhost: [5:1:5], Alias: localhost
            Vhost: [6:1:6], Alias: localhost
            Vhost: [8:1:8], Alias: localhost
            Vhost: [9:1:9], Alias: localhost
            Vhost: [10:1:10], Alias: localhost
            Vhost: [11:1:11], Alias: localhost
            Vhost: [12:1:12], Alias: localhost
            Vhost: [13:1:13], Alias: localhost
            Vhost: [14:1:14], Alias: localhost
            Vhost: [15:1:15], Alias: localhost
            Vhost: [16:1:16], Alias: localhost
            Vhost: [17:1:17], Alias: localhost
            Vhost: [18:1:18], Alias: localhost
            Vhost: [19:1:19], Alias: localhost
            Vhost: [20:1:20], Alias: localhost
            Vhost: [21:1:21], Alias: localhost
            Vhost: [22:1:22], Alias: localhost
            Vhost: [23:1:23], Alias: localhost
            Vhost: [24:1:24], Alias: localhost
            Vhost: [25:1:25], Alias: localhost
            Vhost: [26:1:26], Alias: localhost
            Context: [1:1:1], Context: /ts_webservice_def_x_v1, Status: ENABLED
            Context: [2:1:2], Context: /taskclient_webservice-1.0, Status: ENABLED
            Context: [3:1:3], Context: /objectpartner_webservice_x, _x, Status: ENABLED
            Context: [10:1:12], Context: /dms.importer.admintool_x, Status: ENABLED
            Context: [10:1:13], Context: /dms.importer.delivery_webservice_x, Status: ENABLED
            Context: [11:1:14], Context: /bwb_webservice_x_v1, Status: ENABLED
            Context: [11:1:15], Context: /ts_webservice_op_x_v1, Status: ENABLED
            Context: [12:1:16], Context: /orpa_shortinfo_ws_v2, Status: ENABLED
            Context: [12:1:17], Context: /orpa_details_ws_v1_1, Status: ENABLED
            Context: [13:1:18], Context: /productstructureservice_webservice_x, Status: ENABLED
            Context: [13:1:19], Context: /contract_webservice_x, Status: ENABLED
            Context: [13:1:20], Context: /contract_defa_webservice_x, Status: ENABLED
            Context: [14:1:21], Context: /odsservice_v1, Status: ENABLED
            Context: [15:1:22], Context: /dms.dmsviewer_x, Status: ENABLED
            Context: [15:1:23], Context: /dms.retrieval.contentpresentation_x, Status: ENABLED
            Context: [15:1:24], Context: /dms.doctypeinfo.service_webservice_x, Status: EN[19:1:32], Context: /povws_product_v3_x, Status: ENABLED
            Context: [19:1:33], Context: /povws_product_v2_x, Status: ENABLED
            Context: [19:1:34], Context: /povws_print_v1_x, Status: ENABLED
            Context: [20:1:35], Context: /sanacontractaccesscontrolservice_ws_x, Status: ENABLED
            Context: [21:1:36], Context: /mos_webservice-0.2, Status: ENABLED
            Context: [22:1:37], Context: /pasinterface_webservice_v2_x, Status: ENABLED
            Context: [23:1:38], Context: /mmp, Status: ENABLED
            Context: [23:1:39], Context: /mmp_admin, Status: ENABLED
            Context: [24:1:40], Context: /ub_webservice_v2_x, Status: ENABLED
            Context: [24:1:41], Context: /ub_webgui, Status: ENABLED
            Context: [25:1:42], Context: /activebpel-cert, Status: ENABLED
            Context: [26:1:43], Context: /processinfo_webservice_v2_x, Status: ENABLED
            
            CONFIG / HTTP/1.0
            Content-Length: 98
            User-Agent: ClusterListener/1.0
            Connection: Keep-Alive
            JVMRoute=defaultmetro_u_01&Balancer=defaultmetro&Host=10.32.34.70&Maxattempts=1&Port=8009&Type=ajp
            

             

            1.1.3:

            INFO / HTTP/1.1
            Host: jsplb01x.umobi.mobicorp.test:80
            Content-Length: 0
            User-Agent: ClusterListener/1.0
            Connection: Keep-Alive
            
            HTTP/1.1 200 OK
            Date: Tue, 20 Sep 2011 11:32:59 GMT
            Server: Apache
            Vary: Accept-Encoding
            Keep-Alive: timeout=10
            Connection: Keep-Alive
            Transfer-Encoding: chunked
            Content-Type: text/plain
            1f82
            Node: [1],Name: tariffing_def_x_01,Balancer: tariffing_def,LBGroup: ,Host: 10.32.34.58,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 91
            Node: [2],Name: taskclient_x_01,Balancer: taskclient,LBGroup: ,Host: 10.32.34.173,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 150,Read: 1914106,Transfered: 207588,Connected: 0,Load: 92
            Node: [3],Name: bonusmalus_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.152,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 91
            Node: [4],Name: cmqtis_x_01,Balancer: cmqtis,LBGroup: ,Host: 10.32.35.173,Port: 8009,Type:ax: 61,Ttl: 60,Elected: 153,Read: 10917426,Transfered: 62093,Connected: 0,Load: 90
            Node: [8],Name: beleg_x_01,Balancer: beleg,LBGroup: ,Host: 10.32.35.139,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 92
            Node: [9],Name: applicationlocking_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.35.71,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 87,Read: 161879,Transfered: 62620,Connected: 0,Load: 91
            Node: [10],Name: ecm_import_x_01,Balancer: ecm_import,LBGroup: ,Host: 10.32.34.199,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 4,Read: 1852,Transfered: 1824,Connected: 0,Load: 84
            Node: [11],Name: tariffing_op_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.196,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 306,Read: 8887872,Transfered: 11971680,Conice_x_01,Balancer: odsservice,LBGroup: ,Host: 10.32.34.81,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 13,Read: 31861,Transfered: 6629,Connected: 0,Load: 91
            Node: [15],Name: ecm_retrieval_x_01,Balancer: ecm_retrieval,LBGroup: ,Host: 10.32.34.200,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 613,Read: 2628920,Transfered: 158543,Connected: 0,Load: 88
            Node: [16],Name: defa_x_01,Balancer: defa,LBGroup: ,Host: 10.32.35.110,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 3557,Read: 115225123,Transfered: 187284,Connected: 0,Load: 69
            Node: [17],Name: rdi_x_01,Balancer: rdi,LBGroup: ,Host: 10.32.35.170,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 92
            Node: [18],Name: processinfo_x_01,Balancer: mycluster,LBGroup: ,Host: 10.32.34.179,Port: 8009,Type1,Ttl: 60,Elected: 15,Read: 7640,Transfered: 9313,Connected: 0,Load: 91
            Node: [21],Name: moscontroller_x_01,Balancer: moscontroller,LBGroup: ,Host: 10.32.35.177,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 3,Read: 1011,Transfered: 723,Connected: 0,Load: 90
            Node: [22],Name: pas_interface_v2_x_01,Balancer: pas_interface_v2,LBGroup: ,Host: 10.32.35.144,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 75,Read: 76539,Transfered: 58899,Connected: 0,Load: 91
            Node: [23],Name: mmp_x_01,Balancer: mmp,LBGroup: ,Host: 10.32.35.47,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 82,Read: 873613,Transfered: 285,Connected: 0,Load: 90
            Node: [24],Name: ub_x_01,Balancer: ub,LBGroup: ,Host: 10.32.34.128,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 61,Ttl: 60,Elected: 1809,Read: 18090731,Transfered: 47942,Connected: 0,Loadt
            Vhost: [2:1:2], Alias: localhost
            Vhost: [3:1:3], Alias: localhost
            Vhost: [4:1:4], Alias: localhost
            Vhost: [5:1:5], Alias: localhost
            Vhost: [6:1:6], Alias: localhost
            Vhost: [8:1:8], Alias: localhost
            Vhost: [9:1:9], Alias: localhost
            Vhost: [10:1:10], Alias: localhost
            Vhost: [11:1:11], Alias: localhost
            Vhost: [12:1:12], Alias: localhost
            Vhost: [13:1:13], Alias: localhost
            Vhost: [14:1:14], Alias: localhost
            Vhost: [15:1:15], Alias: localhost
            Vhost: [16:1:16], Alias: localhost
            Vhost: [17:1:17], Alias: localhost
            Vhost: [18:1:18], Alias: localhost
            Vhost: [19:1:19], Alias: localhost
            Vhost: [20:1:20], Alias: localhost
            Vhost: [21:1:21], Alias: localhost
            Vhost: [22:1:22], Alias: localhost
            Vhost: [23:1:23], Alias: localhost
            Vhost: [24:1:24], Alias: localhost
            Vhost: [25:1:25], Alias: localhost
            Vhost: [26:1:26], Alias: localhost
            Context: [1:1:1], Context: /ts_webservice_def_x_v1, Status: ENABLED
            Context: [2:1:2], Context: /taskclient_webservice-1.0, Status:NABLED
            Context: [9:1:11], Context: /applicationlocking_webservice_x, Status: ENABLED
            Context: [10:1:12], Context: /dms.importer.admintool_x, Status: ENABLED
            Context: [10:1:13], Context: /dms.importer.delivery_webservice_x, Status: ENABLED
            Context: [11:1:14], Context: /bwb_webservice_x_v1, Status: ENABLED
            Context: [11:1:15], Context: /ts_webservice_op_x_v1, Status: ENABLED
            Context: [12:1:16], Context: /orpa_shortinfo_ws_v2, Status: ENABLED
            Context: [12:1:17], Context: /orpa_details_ws_v1_1, Status: ENABLED
            Context: [13:1:18], Context: /productstructureservice_webservice_x, Status: ENABLED
            Context: [13:1:19], Context: /contract_webservice_x, Status: ENABLED
            Context: [13:1:20], Context: /contract_defa_webservice_x, Status: ENABLED
            Context: [14:1:21], Context: /odsservice_v1, Status: ENABLED
            Context: [15:1:22], Context: /dms.dmsviewer_x, Status: ENABLED
            Context: [15:1:23], Context: /dms.retrieval.contentpresentation_x, Status: ENABLED
            Context: [15:ext: [19:1:31], Context: /povws_contract_v2_x, Status: ENABLED
            Context: [19:1:32], Context: /povws_product_v3_x, Status: ENABLED
            Context: [19:1:33], Context: /povws_product_v2_x, Status: ENABLED
            Context: [19:1:34], Context: /povws_print_v1_x, Status: ENABLED
            Context: [20:1:35], Context: /sanacontractaccesscontrolservice_ws_x, Status: ENABLED
            Context: [21:1:36], Context: /mos_webservice-0.2, Status: ENABLED
            Context: [22:1:37], Context: /pasinterface_webservice_v2_x, Status: ENABLED
            Context: [23:1:38], Context: /mmp, Status: ENABLED
            Context: [23:1:39], Context: /mmp_admin, Status: ENABLED
            Context: [24:1:40], Context: /ub_webservice_v2_x, Status: ENABLED
            Context: [24:1:41], Context: /ub_webgui, Status: ENABLED
            Context: [25:1:42], Context: /activebpel-cert, Status: ENABLED
            Context: [26:1:43], Context: /processinfo_webservice_v2_x, Status: ENABLED
            
            0
            
            CONFIG / HTTP/1.1
            Host: jsplb01x.umobi.mobicorp.test:80
            Content-Length: 98
            User-Agent: ClusterListener/1.0
            Connection: Keep-Alive
            JVMRoute=defaultmetro_u_01&Balancer=defaultmetro&Host=10.32.34.70&Maxattempts=1&Port=8009&Type=ajp
            
            HTTP/1.1 200 OK
            Date: Tue, 20 Sep 2011 11:32:59 GMT
            Server: Apache
            Content-Length: 0
            Keep-Alive: timeout=10
            Connection: Keep-Alive
            Content-Type: text/plain
            
            • 3. Re: String index out of range: -1
              rhusar

              Hey Yves, just checking, when you were switching the mod_cluster versions in the AS have you also changed the modules on the load balancer? Otherwise I can't seem to reproduce it yet.

              • 4. Re: String index out of range: -1
                yves.p

                Hey Rado

                 

                No I didn't changed the modules in httpd. I left them on 1.1.3 for all my tests.

                The problem occures only with a lot of nodes/contexts, if there are no other nodes everything works fine with 1.1.3. I have a hunch that is a parsing bug when the response from the httpd gets large.

                • 5. Re: String index out of range: -1
                  rhusar

                  Yeah, right, thats useful information too for tracing it, but surely its important that this is happening with 1.1.3 jars + httpd modules which is not good.Please let me know if you come up with a reliabe way to reproduce.

                  • 6. Re: String index out of range: -1
                    yves.p

                    I was able to reproduce the exception reliably. I used up 18 JBoss 7.0.1 standalone server with a variable amount of deployments per server. All JBoss servers and httpd are on the same host. After adding a war to a all servers I restart httpd so the server need to reregister.

                    • 18 servers no deplyoments: works
                    • 18 servers 2 wars: one server can't register an throws the "String index out of range: -1" exception
                    • 18 server 9 wars: only 10 servers can connect.

                     

                    I seams that httpd returns some extra gibrish when taking to the appserver when the number of deplyoments/appserver exeeds a certain limit (as seen in the tcpdump). This extra characters lead to a parsing exception on the appserver. With mod_cluster 1.1.0 on JBoss (which uses http 1.0) everything works fine with 18+ servers and deployments.

                     

                    I attached my jboss 7 standalone config with ears and the httpd config. Hope it helps...

                    • 7. Re: String index out of range: -1
                      yves.p

                      Any news on that problem, has some managed to reproduce it somewhere else?

                      I could try to reproduce it on windows on a completely different system if that's any help.

                       

                      Should I open an issue for it?

                      • 8. Re: String index out of range: -1
                        jfclere

                        That looks like a bug please fill a JIRA.