I did a test about deploy VDB to Teiid. I used 20 thread to create VDB. Teiid server throw below log. I want to know why it is and how to avoid it.
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
How many client connection do you have. What platform. Teiid certainly does not throw too many files for moderate level of usage. However if you are opening lot of connections and holding on to them and doing the large queries (millions of records) there slight possibility. But you would have describe more to about your use case.
It may help to roughly account for things that may use a file handle:
-Each incoming socket connection
-Each outgoing socket connection to a source
-The buffermanager file cache, which is limited to 64 open files by default
-Each read from a Teiid filestore over the previous limit. There will never be more concurrent reads than maxActivePlans
-Any file usage by your sources
We have also seen a similar issue in the past with large jvm sizes (32gigs or more) and some vendor drivers.
Avoiding the issue may include increasing your open file limit, limiting source connections, etc.
Thanks your help.
platform: jboss soa 5.0.1 linux
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 135167
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 135167
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
-Xms1303m -Xmx3303m -XX:PermSize=256m -XX:MaxPermSize=512m -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+HeapDumpOnOutOfMemoryError
This case is used 20 concurrent thread to deploy and undeploy VDB
Duration: 2 hours
I have tested with deploying and undelpoying of VDB.
I found :
When deploy 2 VDB and undeploy 2 VDB, then I execute "lsof" to check. I found teiid server will increase 6 "open file"
java 6780 root 395r IPv4 117035708 TCP 192.168.3.3:59916 (LISTEN)
java 6780 root 416u IPv4 117035704 TCP 192.168.3.3:59915 (LISTEN)
java 6780 root 424u REG 253,0 749935 68519379 /opt/jboss-teiid-5.0.1/jboss-as/server/var/mwm/jboss/server/teiid-server/tmp/teiid/buffer/b_temp-stream_0_4310886018569254268.tmp
java 6780 root 426u REG 253,0 749933 68519380 /opt/jboss-teiid-5.0.1/jboss-as/server/var/mwm/jboss/server/teiid-server/tmp/teiid/buffer/b_temp-stream_0_5857253331971952634.tmp
java 6780 root 427u IPv4 117035721 TCP 192.168.3.3:59917 (LISTEN)
java 6780 root 428u IPv4 117035722 TCP 192.168.3.3:59918 (LISTEN)
I found teiid add 2 open file and 4 listen. Maybe it is caused this issue. Do you think it is leak for teiid?
I met many question about teiid in our endurance test. I want to know whether it is reasonable. In our architecture, we will used many VDB.Maybe the number will over 100. And under the each VDB, it will have another vdb and an oracle. I think this architecture is so large for teiid.
Do you have any advice? Or do you think how many should teiid in the jboss?
Thanks your help.