2 Replies Latest reply on Nov 23, 2010 11:02 AM by timconinxrvp

    recovering from invalid messages, selectively deleting from db

    timconinxrvp

      We use Jboss 5.0 with an oracle db and a standard jboss-messaging deployment.

       

      Irregularly, a queue is unable to start and (therefore) bind itself in JNDI. This has something to do with (I think) corrupt messages which prevent the queue from starting: the exception we see is

      java.sql.SQLException: Invalid column index

       

      The error happens very irregularly (like I said), and we haven't been able to really reproduce it yet. However, it seems to happen when a jboss server is killed dirty while messages are being processed.

       

      Deleting all messages directly in the db (delete from jbm_msg / delete from jbm_msg_ref) resolves the problem, but unfortunately it also removes other queue's messages.

       

      So the question is, either:

      - is there a way to sql-delete only the offending queue's messages, or

      - is there a more elegant way to recover from corrupt messages (remember, the queue does not even start)

        • 1. Re: recovering from invalid messages, selectively deleting from db
          gaohoward

          I think you should find why the messages are corrupted before directly operate on the tables. Those tables JBM_* are not supposed to be manually changed.

           

          A full stack trace will be helpful to find the exact cause.

          • 2. Re: recovering from invalid messages, selectively deleting from db
            timconinxrvp

            It seems that the problem has to do with messages staying in 'delivering' mode:

            When application code called by the MDB throws an exception, it seems that the message does not leave the state 'delivering'. A DLQ has been configured, and the messages end up succesfully over there, but still the original Queue has the problem.

             

            Code-wise, the MDB will only do a

            context.setRollbackOnly();

            when it encounters a JMSException (I hope that's okay).

             

            Trying to remove all messages through the jmx console does not work as 'messages are still being delivered', with the exception

             

            java.lang.IllegalStateException: Cannot remove references while deliveries are in progress (Channel 13360), there are 1
            

             

            And when stopping and starting the MDB (also when starting a server on the same db) we get the exception

             

            java.lang.IllegalStateException: Cannot remove references while deliveries are in progress (Channel 13360), there are 1
            java.sql.SQLException: Invalid column index
             at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
             at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)
             at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:208)
             at oracle.jdbc.driver.OracleResultSetImpl.getObject(OracleResultSetImpl.java:903)
             at org.jboss.resource.adapter.jdbc.WrappedResultSet.getObject(WrappedResultSet.java:750)
             at org.jboss.messaging.core.impl.JDBCPersistenceManager.loadFromStart(JDBCPersistenceManager.java:999)
             at org.jboss.messaging.core.impl.PagingChannelSupport.load(PagingChannelSupport.java:211)
             at org.jboss.jms.server.destination.QueueService.startService(QueueService.java:105)
            
            Starting the Queue, and in fact, the whole messageing subsystem, can then only be done after SQL deleteing the jbm_msg and jbm_msg_ref tables.
            Hope this information helps