-
1. Re: teiid-buffer folder occupied 32 GB space
shawkins Nov 5, 2013 2:24 PM (in response to amitagrawal)Without [TEIID-2714] Buffer defrag not working - JBoss Issue Tracker Teiid will tend to take up the max allocation allowed (as more space is demanded) and stay there. The contents are specific to each start of the server. You are free to delete it when shutdown. We do a full cleanup in startup, but we may want to add a hook somewhere to attempt to clean on shutdown.
-
2. Re: teiid-buffer folder occupied 32 GB space
amitagrawal Nov 5, 2013 2:34 PM (in response to shawkins)Hi Steve,
Thanks for the reply. does this mean that the issue still there in 8.2 and fix is available in 8.6?
we do have a recycle process to restart the jboss every week but I suspect this issue can come up anytime during the week as well, so what is the best approach to mitigate the risk here?
-
3. Re: teiid-buffer folder occupied 32 GB space
shawkins Nov 5, 2013 2:49 PM (in response to amitagrawal)You can configure a smaller space limit for the disk space using max-buffer-space (the default is 50 gigs). But if at any point in time you are actually consuming more than the max, then you'll get some exceptions. Or you'd have to backport TEIID-2714. Otherwise you can safely delete the buffer data when shutdown. And probably an enhancement for us to add a shutdown hook to cleanup would be good and would address many situations except for a server crash.
-
4. Re: teiid-buffer folder occupied 32 GB space
shawkins Nov 6, 2013 8:26 AM (in response to shawkins)> And probably an enhancement for us to add a shutdown hook to cleanup would be good and would address many situations except for a server crash.
Looking at this again there is already a stop hook on the service that should get called on a proper server shutdown.
-
5. Re: teiid-buffer folder occupied 32 GB space
shawkins Nov 7, 2013 9:04 AM (in response to shawkins)TEIID-2714 has been updated to use a system property org.teiid.compactBufferFiles=true to indicate that space should be more actively minimized in the buffer files. This should be only a small overhead and will lead to more filesystem fragmentation.
-
6. Re: teiid-buffer folder occupied 32 GB space
amitagrawal Nov 7, 2013 9:38 AM (in response to amitagrawal)Hi Steve,
does below look goood to you for setting the disk space of 15GB? we are seeing 100% usage on a daily basis now and need to put limit on disk space usage.
<buffer-service use-disk="true" processor-batch-size="512" connector-batch-size="1024"
max-processing-kb="-1" max-reserve-kb="-1" max-file-size="2048" max-buffer-space="15260" max-open-files="64"
memory-buffer-space="-2" memory-buffer-off-heap="true" max-storage-object-size="102400" inline-lobs="false"/>
-
7. Re: teiid-buffer folder occupied 32 GB space
shawkins Nov 7, 2013 1:17 PM (in response to amitagrawal)That will cap our disk usage, but if at any point in time (lots of lobs, several large sorts running concurrently, etc.) that is exceeded, then you'll see an exception.
-
8. Re: teiid-buffer folder occupied 32 GB space
amitagrawal Nov 13, 2013 2:08 PM (in response to shawkins)Hi Steve,
I added below entry in my standalone-teiix.xml file to cap the disk usaget but it seems it doens not honour the size. I am still able to see my buffer size around 150 MB and no error in the log.
<subsystem xmlns="urn:jboss:domain:teiid:1.0">
<buffer-service max-buffer-space="1"/>
</subsystem>
-
9. Re: teiid-buffer folder occupied 32 GB space
shawkins Nov 13, 2013 3:03 PM (in response to amitagrawal)Is that specifically in the teiid buffer files or does it include all of data/temp?
-
10. Re: teiid-buffer folder occupied 32 GB space
amitagrawal Nov 13, 2013 3:35 PM (in response to shawkins)150 MB file size is just for \teiid-buffer\buffer folder.
-
11. Re: teiid-buffer folder occupied 32 GB space
shawkins Nov 13, 2013 5:50 PM (in response to amitagrawal)Can you provide which files (and do they coincide with that run of the server) are consuming space? And can you check via the CLI or elsewhere that the buffer-service subsystem does have the expected setting for the max space.
-
12. Re: teiid-buffer folder occupied 32 GB space
amitagrawal Nov 15, 2013 4:26 PM (in response to shawkins)we also noticed that the default 50GB limit is also not capped and the process keep consuming more space, in our DEV environment, it conusmed 56 GB and after that file system got filled 100%.
in my lcoal system with 1MB max buffer space, the below directory under \tmp\teiid-buffer\buffer\ occupied the space.
b5 - 165 MB
b4 - 62MB
b3 - 8kb
-
13. Re: teiid-buffer folder occupied 32 GB space
shawkins Nov 15, 2013 8:14 PM (in response to amitagrawal)Is this on 8.2 or 8.6? If it's 8.2 more than likely you are seeing [TEIID-2357] max buffer space is not honored - JBoss Issue Tracker
-
14. Re: teiid-buffer folder occupied 32 GB space
amitagrawal Nov 15, 2013 8:23 PM (in response to shawkins)
it is 8.2. what are the options we have in 8.2 to resolve this issue?