9 Replies Latest reply on Jun 20, 2013 10:48 AM by rhauch

    large numbers of references

    jonathandfields

      Will Modeshape's implementation of references scale to thousands or even millions of references to the same node? Or would it be better in this case to just store the referenced node's  path or identifier in a property?

       

      Thanks!

        • 1. Re: large numbers of references
          rhauch

          No, at this time it is probably not feasible to have thousands of REFERENCE properties to the same node. You could try it, but I don't think performance will be acceptable.

           

          The reason is that ModeShape currently stores an internal back-reference from the referenced node to the referring node, since the JCR API requires the ability for a node to return all nodes that have a REFERENCE to it. So having a single node store thousands of back-references will not really be terribly efficient, since all of that information is stored within the node's JSON document inside Infinispan. (We've implemented a segmentation of child nodes to handle large numbers of children, but not large numbers of back-references.)

          • 2. Re: large numbers of references
            jonathandfields

            Would it be appropriate to submit a JIRA enhancement request for such support?

            • 3. Re: large numbers of references
              rhauch

              Sure, though I'm not sure it'd be terribly high on our priority list. Of course, we'd welcome any contributions. The segmented children is fairly complicated, but we could walk you through it to get you going.

              • 4. Re: large numbers of references
                jonathandfields

                Thanks, but I'll leave it as-is for now. It is easy enough to accomplish what I am trying to do by storing identifiers in properties. 

                 

                It's too bad the JCR spec does not support a PropertyType.IDENTIFIER  that works the same as PropertyType.PATH, only by storing the more stable identifier than the path to the referenced node.

                 

                Essentially what  I am trying to accomplish is to know when to render a link in a user interface, instead of just displaying the Property's value. Basing that decision on the Propertie's type would be the most general way to do it, but for now I will have to know that specific Properties of type STRING contain identifiers to other nodes.

                • 5. Re: large numbers of references
                  rhauch

                  Actually, I've thought previously about a UNIDIRECTIONAL_REFERENCE and UNIDIRECTIONAL_WEAKREFERENCE that basically would work the same way as you suggest (internally, for a REFERENCE ModeShape does store the identifier of the referrenced node). Essentially they are JCR references in every sense except that there is no support for back-references. (BTW, they'd also be faster to persist during save.)

                   

                  We could add this to ModeShape, but unfortunately the JSR-333 public draft was recently published and so it's closed to new features like this.

                  • 6. Re: large numbers of references
                    jonathandfields

                    I think this would be a useful alternative to the PropertyType.PATH due to the instability of a path, and to (WEAK)REFERENCEs due to the performance limitations related to the back pointers.  Too bad the next JCR spec is closed.

                     

                    I can move ahead without them, but would be happy to submit the JIRAs to add unidirectional references and to improve the back pointer performance for tracking purposes. Please just let me know if you'd like me to do that. I'd love to contribute but unfortunately other priorities prevail at the moment.

                     

                    Thanks!

                    • 7. Re: large numbers of references
                      rhauch

                      Yeah, please open an enhancement request.

                      • 8. Re: large numbers of references
                        jonathandfields

                        MODE-1969

                        MODE-1970

                         

                        Please adjust as desired.

                         

                        Thanks.

                        • 9. Re: large numbers of references
                          rhauch

                          Thanks!