8 Replies Latest reply on Jul 2, 2010 5:23 PM by rhauch

JCR.SQL2 Join error

manuel.gentile Jun 28, 2010 11:09 AM

Maybe there is a mistake in the class JoinComponent.

at line 359 the parameters og Joinable object have to be swapped like in the folliwing code...

Moreover I have some problems with casting .... (BasicPath is not castable to Location reference!)

else if (condition instanceof ChildNodeJoinCondition) {
            ChildNodeJoinCondition joinCondition = (ChildNodeJoinCondition)condition;
            String childSelectorName = joinCondition.getChildSelectorName().getName();
            if (left.getColumns().hasSelector(childSelectorName)) {
                // The child is on the left ...
                return new Joinable() {
                    public boolean evaluate( Object parentLocation,Object childLocation
                                              ) {
                    
                         Path childPath=null;
                         Path parentPath =null;
                         if (childLocation instanceof BasicPath)
                         {
                              childPath =((BasicPath) childLocation).getCanonicalPath();
                         } else
                              childPath = ((Location)childLocation).getPath();
                    
                         if (parentLocation instanceof BasicPath)
                         {
                              parentPath =((BasicPath) parentLocation).getCanonicalPath();
                         } else
                              parentPath = ((Location)parentLocation).getPath();

At the end matking a query like this

select lom.* from [lom:Metadata] as lom JOIN [lom:LangString] as lang ON ISCHILDNODE(lang,lom)

in which lom:LangString is a child node of lom:Metadata type I have an error like this

There has been an error processing your command
  null
java.lang.ArrayIndexOutOfBoundsException
     at java.lang.System.arraycopy(Native Method)
     at org.modeshape.graph.query.process.JoinComponent$2.merge(JoinComponent.java:217)
     at org.modeshape.graph.query.process.NestedLoopJoinComponent.execute(NestedLoopJoinComponent.java:69)
     at org.modeshape.graph.query.process.ProjectComponent.execute(ProjectComponent.java:48)
     at org.modeshape.graph.query.process.QueryProcessor.execute(QueryProcessor.java:93)
     at org.modeshape.graph.query.QueryEngine.execute(QueryEngine.java:111)
     at org.modeshape.jcr.RepositoryQueryManager$SelfContained.query(RepositoryQueryManager.java:361)
     at org.modeshape.jcr.JcrQueryManager$JcrQuery.execute(JcrQueryManager.java:379)

Thanks

1. Re: JCR.SQL2 Join error

rhauch Jun 28, 2010 12:12 PM (in response to manuel.gentile)

Excellent catch! Would you mind logging a JIRA issue for this bug? We're a day or two away from releasing 2.0, so it'd be great to get this into the 2.0 release!

Patches are also welcome, too. So if you already have a fix (and preferably a new test case, perhaps in JcrQueryManagerTest), just upload the patch directly to the JIRA issue.

Great job, Manuel!
Actions

2. Re: JCR.SQL2 Join error

manuel.gentile Jun 29, 2010 5:17 AM (in response to rhauch)

I'm trying to solve this bug but I think there is something wrong also in the following code....

I wonder if the goal is to merge two array (left and right tuples) into result???

If it is, the System.arraycopy is wrong!! source and destination array are swapped???

Can you explain this code???

return new TupleMerger() {
                /**
                 * {@inheritDoc}
                 * 
                 * @see org.modeshape.graph.query.process.JoinComponent.TupleMerger#merge(java.lang.Object[], java.lang.Object[])
                 */
                public Object[] merge( Object[] leftTuple,
                                       Object[] rightTuple ) {
                    Object[] result = new Object[joinTupleSize]; // initialized to null
                    // If the tuple arrays are null, then we don't need to copy because the arrays are
                    // initialized to null values.
                    if (leftTuple != null) {
                        // Copy the left tuple values ...
                        System.arraycopy(result, 0, leftTuple, 0, leftColumnCount);
                        System.arraycopy(result, startLeftLocations, leftTuple, leftColumnCount, leftLocationCount);
                        if (leftScores) {
                            System.arraycopy(result, startLeftScores, leftTuple, leftLocationCount, leftScoreCount);
                        }
                    }
                    if (rightTuple != null) {
                        // Copy the right tuple values ...
                        System.arraycopy(result, leftColumnCount, rightTuple, 0, rightColumnCount);
                        System.arraycopy(result, startRightLocations, rightTuple, rightColumnCount, rightLocationCount);
                        if (rightScores) {
                            System.arraycopy(result, startRightScores, rightTuple, rightLocationCount, rightScoreCount);
                        }
                    }
                    return result;
                }
            };
        }
        // There are no full-text search scores ...
        return new TupleMerger() {
            /**
             * {@inheritDoc}
             * 
             * @see org.modeshape.graph.query.process.JoinComponent.TupleMerger#merge(java.lang.Object[], java.lang.Object[])
             */
            public Object[] merge( Object[] leftTuple,
                                   Object[] rightTuple ) {
                Object[] result = new Object[joinTupleSize]; // initialized to null
                // If the tuple arrays are null, then we don't need to copy because the arrays are
                // initialized to null values.
                if (leftTuple != null) {
                    // Copy the left tuple values ...
                    System.arraycopy(result, 0, leftTuple, 0, leftColumnCount);
                    System.arraycopy(result, startLeftLocations, leftTuple, leftColumnCount, leftLocationCount);
                }
                if (rightTuple != null) {
                    // Copy the right tuple values ...
                    System.arraycopy(result, leftColumnCount, rightTuple, 0, rightColumnCount);
                    System.arraycopy(result, startRightLocations, rightTuple, rightColumnCount, rightLocationCount);
                }
                return result;
            }
        };

3. Re: JCR.SQL2 Join error

rhauch Jun 30, 2010 10:56 AM (in response to manuel.gentile)

Sorry for the delay. I've been looking into this problem, and it turns out to be a bit more complicated than I'd first though.

The original problem you mention looks to be similar to MODE-772. And what I said earlier was incorrect: there never should be a BasicPath for the location. The fix is not to change how the cast is made, but the correct fix is to make sure BasicPath never appears in the tuple. I have a local fix, but as you point out, that fix leads to another issue with the JOIN logic, which was indeed very wrong.

Now, surprisingly, this logic is not always used on queries with JOIN clauses. You see, because of how JCR maps node types into tables, a node actually can appear in multiple tables if that node is considered an instance of more than one node type. And to get the properties of a node in a single result set, where those properties are defined on multiple types, you have to do what we call an identity equijoin. Here's a JCR-SQL2 example:

SELECT * FROM a JOIN b ON ISSAMENODE(a,b) ...

and a JCR-SQL example:

SELECT * FROM a,b WHERE a.jcr:path = b.jcr:path ...

Queries of this form are very common (and interestingly these are the only kinds of joins allowed in JCR-SQL). But because of the way ModeShape organizes its indexes, ModeShape actually optimizes these queries (even when there are multiple identity equi-joins involved) into a form that has no join but is instead just a single, optimized SELECT against our indexes. These queries all work, because they're executed without using the faulty JOIN logic.

Even some queries that involve a single ISCHILDNODE or ISDESCENDANTNODE criteria can be optimized into a form without a JOIN. We have a number of these kinds of tests, and they all work fine, too.

However, I suspect most queries involving ISCHILDNODE or ISDESCENDANTNODE criteria will not be optimized and thus will be processed as a true JOIN. As you might have guessed, we have no such queries in our tests, which is why the faulty JOIN logic went undetected for so long.

I have a fix for the faulty join logic, too, but this led me to discover another problem in the way our query plan optimizer is manipulating the JOIN tree, resulting in an incorrect plan that cannot be executed. This is where it gets tricky, and this is what I've been looking into for the last few days.

Suffice to say that this is a critical issue that will block the 2.0 release, and I've logged it as MODE-802. I will update this thread when I find out more or have a solution.
Actions
4. Re: JCR.SQL2 Join error

rhauch Jun 30, 2010 10:50 PM (in response to rhauch)

Okay, I've come up with a patch for MODE-802 and MODE-772 (with additional test cases and the fixes) and committed it to trunk. The full and very detailed explanation of the problem and the fix can be found on a comment on MODE-802.

Manuel, I'd appreciate you taking a look at trunk to see if this resolves the issue for you. If so, please reply here or add a comment to MODE-802.

Thanks!
Actions

5. Re: JCR.SQL2 Join error

manuel.gentile Jul 1, 2010 5:43 AM (in response to rhauch)

I'm sorry ! there is a new exception whe I try the following query

select * from [lom:Metadata] as lom join [lom:LangString] as lang on isdescendantnode(lang,lom)


 
 
There has been an error processing your command
  org.modeshape.jcr.query.qom.JcrJoin cannot be cast to org.modeshape.jcr.query.qom.JcrSource
java.lang.ClassCastException: org.modeshape.jcr.query.qom.JcrJoin cannot be cast to org.modeshape.jcr.query.qom.JcrSource
at org.modeshape.jcr.query.JcrSql2QueryParser.query(JcrSql2QueryParser.java:141)
at org.modeshape.jcr.query.JcrSql2QueryParser.query(JcrSql2QueryParser.java:1)
at org.modeshape.graph.query.parse.SqlQueryParser.parseQuery(SqlQueryParser.java:558)
at org.modeshape.graph.query.parse.SqlQueryParser.parseQueryCommand(SqlQueryParser.java:511)
at org.modeshape.graph.query.parse.SqlQueryParser.parseQuery(SqlQueryParser.java:504)
at org.modeshape.jcr.JcrQueryManager.createQuery(JcrQueryManager.java:140)
at org.modeshape.jcr.JcrQueryManager.createQuery(JcrQueryManager.java:108)
at it.cnr.itd.pa.freeloms.modeshape.service.SearchService.test(SearchService.java:92)
at it.cnr.itd.pa.freeloms.client.ConsoleInput.displayNavigationMenu(ConsoleInput.java:257)
at it.cnr.itd.pa.freeloms.client.ConsoleInput$1.run(ConsoleInput.java:126)
at java.lang.Thread.run(Thread.java:619)

6. Re: JCR.SQL2 Join error

rhauch Jul 1, 2010 2:02 PM (in response to manuel.gentile)

You're running against trunk? The line numbers don't seem to coordinate with the source (prior to my latest revision of rev1921). Would you mind verifying which SVN revision you're running against?

Now, as I mentioned, I did just commit two fixes (rev 1921 and rev1922) to correct some errors in two of the QOM implementation classes (e.g., JcrOr and JcrNot were not implementing the JcrConstraint interface, and JcrArithmeticOperation was not implementing JcrDynamicOperand) causing a class cast exception similar to the one you reported. Plus, these revisions add several new test cases. One of these test cases, JcrSql2QueryParserTest.shouldParseDescendantNodeJoinWithNoCriteria(), even tests your exact query without a parsing error, and another test case, JcrQueryManagerTest.shouldBeAbleToCreateAndExecuteSqlQueryWithDescendantNodeJoinAndColumnsFromBothSidesOfJoin(), successfully executes a similar query against a real Repository instance.

Thanks for helping out with this! Hopefully once this issue is resolved (and a few documentation updates are made), we're ready for the 2.0 release.
Actions
7. Re: JCR.SQL2 Join error

manuel.gentile Jul 2, 2010 5:19 PM (in response to rhauch)

Great work Randall! It works like a charm!!!!

Thanks !
Actions
8. Re: JCR.SQL2 Join error

rhauch Jul 2, 2010 5:23 PM (in response to manuel.gentile)

Great to hear! And thanks, Manuel, for taking the time to report this and verify the fixes!
Actions

Go to original post