Hi Bela,
Thanks for all the suggestions. And I tried various tests in different environments. It seems the performance numbers from PerfTests is more dependent on the JVM and its configuration based on the running system, and less on the network configuration. The following is a summary of my tests.
Basically, I run the tests in two different environments. The first one is:
- Isolated network with 4 physical machines with dual quad core Intel Xeon E5450 @ 3.00 GHz
- 32-bit Linux, 32-bit JVM
The second one is like:
- A network with a lot of machines (physical ones as well as VMs). My test is on 2 VMs with single core Intel Xeon E5450 @ 3.00 GHz
- 64-bit Linux, 64-bit JVM
Regardless of the environments, they all have:
- 4G RAM
- 1G ethernet card
- connected to 1G switch
- traffic prioritazation/flow control/rate limiting all turned off on the switch (seems that's the default settings)
Summery of the tests:
- Regardless of setting (autoneg=on, rx=off, tx=off) or (autoneg=off, rx=on, tx=on), the data for iperf are pretty close for either setting on both environments. For TCP, i can drive it up to 940 Mbits/sec, for UDP up to 740 Mbits/sec (a little lower for VMs in non-isolated env, around 710 Mbits/sec).
- Regardless of setting (autoneg=on, rx=off, tx=off) or (autoneg=off, rx=on, tx=on), the data for PerfTest are pretty close. However, the JVM configuration makes a huge difference in both environments
- For environment 1, with the JVM configuration in jgroups.sh, I could drive up to 83.43MB/sec for TCP, and 70.09MB/sec for UDP (NOTE: I got java.lang.OutOfMemoryError for TCP test though).
- For environment 2, originally I was using -server -Xmx600M -Xms400M -XX:+UseParallelGC -XX:+AggressiveHeap -XX:CompileThreshold=100 -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=31 -Djgroups.bind_addr=${MYTESTIP_1} -Dcom.sun.management.jmxremote -Dresolve.dns=false, I could reach up to for 59.4MB/sec UDP, but only around for 2MB/sec TCP. When I changed to the settings in jgroups.sh, it's down to 5.43MB/sec for UDP, and up to 4.45MB/sec for TCP
The last thing to mention is that I don't have the ReplicationTimeout error on environment 1 for my Infinispan test originally.
Does the above data make sense at all? If the JVM options make a big deal in JGroups performance, do you have guidelines on how to configuration it based on system settings? And is this related to the ReplicationTimeout error I got?
Thanks in advance.
Lin