6 Replies Latest reply: Jul 24, 2012 2:23 AM by Jean-Frederic Clere RSS

routing table lookup performance

Alexey Syomichev Newbie

I recently noticed that as the population of instances behind mod_cluster grows, request routing takes significantly longer time. To give a rough order of magnitude, I am talking about latencies on the order of 100-200ms for pools of about 200+ workers. Parallel requests still scale well (i.e. several request loops running in parallel show the same request rate as a single one), but latency of 200 ms translates into a single-client throughput of 5 HTTP transactions per second, which is alarmingly low.

 

After some digging it became apparent that the time is mostly spent accessing host and context information in the routing table in shared memory. I am talking about host_storage->read_host() and context_storage->read_context() in get_balancer_by_node() (mod_proxy_cluster.c) It looks like reading from shared memory is quite slow and on top of that it is done repeatedly (something close to O(n^2) to the number of  entries).

 

Has someone observed it before? Is the anything in the works to improve?

 

My environment: linux 2.6.18 x86_64, httpd 2.2.15, mod_cluster 1.1.0.Final, Tomcat 6.0.20

 

Thanks,
--Alexey