1 2 Previous Next 16 Replies Latest reply on Aug 14, 2007 8:49 AM by aquest

    Hypersonic localDB.data file size limit

    mulicheng

      We have been running jboss 3.0.x for quite some time without any problems. Switching to 3.2.3 we have a JMS problem. I don't know if there is a configuration file change or if our application is doing something wrong.

      The problem is that in {SERVER_HOME}/data/hypersonic, the localDB.data file continuously grows. It never seems to shrink. We send a LOT of Messages to an MDB and the messages are persisted temporarily until handled. That seems to work just fine but the localDB.data keeps growing even when the messages are finished being handled. The file eventually reaches about 2.0 gigs. At that point, JBoss starts throwing lots of exceptions.. example:

      Could not store message: 11175 msg=11166 hard NOT_STORED
      PERSISTENT queue=QUEUE.EmailQueue priority=4 hashCode=20379376; - nested throwable: (java.sql.SQLException: S1000 General error java.lang.NullPointerException in statement [INSERT INTO JMS_MESSAGES (MESSAGEID, DESTINATION, MESSAGEBLOB, TXID, TXOP) VALUES(11166,
      .... .... ...

      The only solution has been to kill the jboss processes, delete the hypersonic directory, and then start again, letting hypersonic create a new empty database.

      Is there any way to automatically shrink the database?

      We are using the default configuration without any customization other than deploying our application. I can connect to the hypersonic db and verify that there are no records in the JMS_MESSAGES table but the file is still ever growing.
      OS: Linux 2.4.19 kernel
      JDK: Blackdown jdk 1.4.1
      JBoss: 3.2.3

      Thanks for any input.

        • 1. Re: Hypersonic localDB.data file size limit
          starksm64

          Then is the data coming from something else? Look at the localDB.script to see what other data is being committed. If you have an example mdb testcase that shows the ever expanding file post it in a bug report ot sourceforge:
          http://sourceforge.net/tracker/?group_id=22866&atid=376685

          • 2. Re: Hypersonic localDB.data file size limit
            mulicheng

            I'll work on something reproducable. In the mean time, I noticed that it doesn't always grow with every message. In fact most of them seem to be processed normally. There doesn't appear to be any other data in the database. ie: select count(*) from JMS_MESSAGES returns 0. The localDB.script file shows pretty much what I'd expect: inserts and deletes for the Message Queue persisted messages and their transactions.

            The fact remains however, that the data file is currently 1.6 gigs and yesterday it was 1.5 gigs. I anticipate that Monday or Tuesday I'll have to fix it again.

            A side note, the localDB.backup is only 7.6 Megs.

            • 3. Re: Hypersonic localDB.data file size limit
              peter_f

              Try issue a 'SHUDOWN COMPACT' command to cleanslate Your db and se if that is helping You with regards to the different file sizes ...

              • 4. Re: Hypersonic localDB.data file size limit
                genman


                I created a special MBean that would issue "SHUTDOWN COMPACT" when the server got shut down. You can also issue a "CHECKPOINT" command every X minutes through a timer.

                http://sourceforge.net/tracker/index.php?func=detail&aid=844654&group_id=22866&atid=376685

                • 5. Re: Hypersonic localDB.data file size limit
                  jbossy

                  My localDB.script grows huge, was there ever a way to deal with the evergrowing size of this file without being forced to shutdown the JBoss instance? Can someone point me to some documentation on this, I have bought some JBoss books and docs already? Is there some rolling log like feature that I can use?

                  Thanks!

                  • 6. Re: Hypersonic localDB.data file size limit
                    jbossy

                     

                    "jbossy" wrote:
                    My localDB.script grows huge, was there ever a way to deal with the evergrowing size of this file without being forced to shutdown the JBoss instance? Can someone point me to some documentation on this, I have bought some JBoss books and docs already? Is there some rolling log like feature that I can use?

                    Thanks!


                    BTW, I am running medium size 1.3MB bytes messages on JB4DR4.

                    • 7. Re: Hypersonic localDB.data file size limit
                      genman


                      http://hsqldb.sourceforge.net/doc/hsqlSyntax.html#SET%20LOGSIZE

                      The .script file is cleaned up when it hits 200 MB. You can set this size lower if you want.

                      • 8. Re: Hypersonic localDB.data file size limit

                        Hi,
                        I have the same problem, The localDB.data file grows with no limit. Has anyone solved this yet ?
                        How can I invoke CHECKPOINT / SHUTDOWN operation while my server is running?

                        (I'm using jboss 3.2.5)

                        • 9. Re: Hypersonic localDB.data file size limit

                          Hi,

                          we found the same problem on our systems. I did some "tests" and found out, that the hypersonic seems to work pretty smooth, even if it has do do a lot of work. (Sent about some 800.000 messages as fast as the server could, followed up by some other 100.000 messages during the following hours.)
                          There war no problem, until I started some other programs to consume up all physical and virtual memory on my linux-box. (started reading a pdf, wrote it to a vector, read it again, put it in the same vector, created a new vector if an OutOfMemoryException came, ... startet this program some times with Xmx and Xms values between 128MB and 512MB).
                          When the box had 1 Gig RAM used and 2 Gig swap, still eyerthing worked fine. I had a load above 20 and CPU-load 100%. From now on, the .data-file startet to grow... I hat about 100 MB in the .script-file, the .data-file was at ~30MB.
                          Now I stopped the memory-consuming programs and kept looking what happend. Well, the .data-file kept growing. Not that fast, but it did not shrink.

                          Maybe someone can make a more appropriate test-case with this scenario, I also think, that there is no need for such a brutal load. I think it is more important to force the machine to swap.

                          greetz,

                          Christian

                          • 10. Re: Hypersonic localDB.data file size limit
                            freddt

                            1. An automatic checkpoint should take place by default when the .log file reaches 200MB. This will reduce the size of the .log file to a minimum but will not reduce the size of the .data file. (In version 1.7.1, the .script file was used for logging, instead of the currently used .log file). However, there was an issue in HSQLDB 1.7.2.4 (which has been resolved in later revisions) with automatic checkpoints.

                            2. The .data file keeps track of up to 1000 recently deleted rows and uses the empty space for new rows. When more rows are deleted, it abandons the old list. As a result, whenever a very large number of rows are inserted and then deleted, the .data file grows.

                            3. From 1.7.2.x a new command, CHECKPOINT DEFRAG, is available to reduce the size. This takes the database engine offline for the duration of the operation. Any queries will be queued until the new files have been written out and the old ones deleted. (You need temporary disk space equivalent to the existing file sizes for this operation).

                            4. You can reduce the amount of memory used by HSQLDB by setting the cache_scale property to a smaller value (needs a restart). The current defaults allow up to 50MB of data to stay in memory. Reducing the cache_scale to 12 will take this down to 12.5 MB.

                            5. Using operating system swap space for Java programs is very OS and JVM implementation dependent. In the worst case scenario, the garbage collector will continually bring back the swapped-out blocks into memory and slow down the operation of the app server.

                            6. With extensive improvements made to HSQLDB leading to 1.7.2.11 and 1.7.3.1, the engine is now extremly reliable. I would like some feedback on the usage pattern under heavy load, so that we can allow a larger list of empty disk spaces to be kept and reduce the growth of the .data file.

                            Fred Toussi
                            Maintainer, HSQLDB Project

                            • 11. Re: Hypersonic localDB.data file size limit
                              freddt

                              Further to my last message, I have just uploaded the latest 1.8.0 RC3. Go to http://hsqldb.sf.net home page to download the file. (Note this is a Release Candidate and should be used only for testing)

                              With this version the size of the .data file for HSQLDB goes up only by a small fraction of the previous versions for each 200MB worth of logged statements.

                              Also, when the empty spaces in the .data file reach 200MB and the .log file also reaches 200MB and forces an automatic checkpoint, a defrag is automatically performed. So the .data file size will not have more than an average 200MB wasted space and its size will reflect this.

                              As a result of all this, you should be able to run a production JBoss
                              instance indefinetly without any problems.

                              Tests will continue with the 1.8.0 Release Candidates. The final release is scheduled for March.

                              If you test this version, please report your finidings to the HSQLDB user channels via http://hsqldb.sf.net

                              Fred Toussi
                              Maintainer, HSQLDB Project

                              • 12. Re: Hypersonic localDB.data file size limit
                                yarongu

                                Thanks for the detailed and meaningful answer.

                                Yaron G.

                                • 13. Re: Hypersonic localDB.data file size limit
                                  freddt

                                  An update on my earlier posts (a few months ago).

                                  HSQLDB 1.8.0 and bug fix releases have been out for a while.

                                  You can use a release version of HSQLDB (which you can get from http://hsqldb.org), with recent version of JBoss. You may need to make change to JBoss config files or to HSQLDB. AFAIK, these are:

                                  - Issue with a query in JBoss that no longer works (this has apparently been fixed in latest JBoss updates). You need to modify the query. See:
                                  http://sourceforge.net/forum/forum.php?thread_id=1333514&forum_id=73674

                                  - HSQLDB is by default compiled to use some java.nio classes. There may be issues with NFS volumes. Also, memory use is higher than without nio. You should recompile HSQLDB with JDK 1.3.x to avoid such cases. Jars previously supplied with JBoss were recompiled in this way.

                                  - In order to benefit from (a) 8 GB maximum database size, and (b) automatic defrag, you need to set the relevant properties of the HSQLDB database when the database is empty. See the HSQLDB docs on these settings.

                                  Fred Toussi
                                  Maintainer, HSQLDB Project

                                  • 14. Re: Hypersonic localDB.data file size limit
                                    jeffdo666

                                    I'm using JBoss 4.0.5 and I use the following localDB.properties:

                                    #HSQL Database Engine
                                    #Wed Jan 17 15:50:01 EST 2007
                                    hsqldb.script_format=0
                                    runtime.gc_interval=0
                                    sql.enforce_strict_size=false
                                    hsqldb.cache_size_scale=8
                                    readonly=false
                                    hsqldb.nio_data_file=true
                                    hsqldb.cache_scale=14
                                    version=1.8.0
                                    hsqldb.default_table_type=memory
                                    hsqldb.cache_file_scale=8
                                    hsqldb.log_size=20
                                    modified=no
                                    hsqldb.cache_version=1.7.0
                                    hsqldb.original_version=1.8.0
                                    hsqldb.compatible_version=1.8.0

                                    However, I am still seeing the .data file growing without shrinking. I know there is a SET CHECKPOINT DEFRAG cmd, but is there some way you can set in this in the localDB.properties file?


                                    1 2 Previous Next