1 2 3 4 Previous Next 53 Replies Latest reply: Sep 20, 2012 1:24 PM by Jesper Thorhauge RSS

Data sources in EAR on AS7?

henk de boer Master

Since the way to configure data sources has changed significantly in AS 7 (see http://community.jboss.org/wiki/DataSourceConfigurationinAS7), I wonder whether it will still be possible to put such a definition inside an EAR.

  • 1. Data sources in EAR on AS7?
    henk de boer Master

    Does anyone already know the answer to this?

  • 2. Data sources in EAR on AS7?
    Jesper Pedersen Master

    Put your datasource definition in standalone.xml or domain.xml. EAR deployment won't be supported.

  • 3. Data sources in EAR on AS7?
    henk de boer Master

    Jesper Pedersen wrote:

     

    Put your datasource definition in standalone.xml or domain.xml. EAR deployment won't be supported.

     

    I'm very sorry to hear this. Any rationale? Is there a technical reason for this?

     

    We very much depend on this functionality. Our EARs are structured such that they embed all the data sources for the X staging platforms that we deploy to. Based on an environment parameter that parameterizes jboss-app.xml, the right .xml file containing the right datasource is chosen when JBoss starts up.

     

    This allows us to build fully contained EAR files that we can deploy to generic JBoss AS instances on every stage.

     

    I know this is not the only way to do things, but I also know that quite a number of clients use this exact same setup. There are also multiple messages in the forum where people ask about this (and not only about data sources, but also about JMS queues, etc).

     

    Cutting this very useful feature out of JBoss would be nothing less than a major disaster for us and a number of our clients.

     

    At any length you have to support data sources embedded in the EAR, because the Java EE 6 spec demands this. It will then be via the simpler Java EE 6 syntax for this, and not the richer JBoss syntax but still. Given that you already have to support the Java EE 6 version, it seems it would not be incredible hard to keep supporting the JBoss specific one, would it?

  • 4. Data sources in EAR on AS7?
    David Lloyd Master

    Jesper Pedersen wrote:

     

    Put your datasource definition in standalone.xml or domain.xml. EAR deployment won't be supported.

     

    Are you sure this is correct?  I thought we were going to support this (but with only limited management capabilities of course).

  • 5. Data sources in EAR on AS7?
    Jason Greene Master

    You can in fact use the spec described @DataSourceDefinition (data-source in xml) to do this. However this uses JDBC directly, since it directly references JDBC classes, and bypasses the JCA layer. So the disadvantage here is that it is an "unmanaged" data source. Anything in the standalone.xml/domain.xml is "managed" and can be added/removed on the fly, monitored, etc.

  • 6. Data sources in EAR on AS7?
    henk de boer Master

    Jason Greene wrote:

     

    You can in fact use the spec described @DataSourceDefinition (data-source in xml) to do this. However this uses JDBC directly, since it directly references JDBC classes, and bypasses the JCA layer. So the disadvantage here is that it is an "unmanaged" data source. Anything in the standalone.xml/domain.xml is "managed" and can be added/removed on the fly, monitored, etc.

     

    Hmmm, does "unmanged" in this context also means connections aren't pooled and perhaps aren't suited for use in XA transactions?

     

    In that case the spec described datasource wouldn't really work for what I need.

     

    I could very well live without management capabilities like adding/removing and monitoring, but it's fairly crucial for us that datasources can be embedded inside the EAR. We have used this capability of JBoss AS for years now and pretty much our entire deployment strategy depends on it.

  • 7. Data sources in EAR on AS7?
    Jason Greene Master

    henk de boer wrote:

     

    Jason Greene wrote:

     

    You can in fact use the spec described @DataSourceDefinition (data-source in xml) to do this. However this uses JDBC directly, since it directly references JDBC classes, and bypasses the JCA layer. So the disadvantage here is that it is an "unmanaged" data source. Anything in the standalone.xml/domain.xml is "managed" and can be added/removed on the fly, monitored, etc.

     

    Hmmm, does "unmanged" in this context also means connections aren't pooled and perhaps aren't suited for use in XA transactions?

    By unmanaged I just mean that an administrator couldn't control it which is important to some users.

     

    It all depends on what jdbc Resource class you point the annotation/xml too. If you point it to a driver XADatasource you get XA, to a PooledDataSource you get connection pooling. Let us know if you think the spec mechanisms do not meet your needs, and we can start a discussion on the forums about what features are needed. You can actually try out DataSourceDefinition annotation in the beta release right now

  • 8. Data sources in EAR on AS7?
    henk de boer Master

    Jason Greene wrote:

     

    It all depends on what jdbc Resource class you point the annotation/xml too. If you point it to a driver XADatasource you get XA, to a PooledDataSource you get connection pooling.

     

    Okay, I had some problems with this in JBoss AS 6, but I'll try again in AS7. Nevertheless, this does mean my DB driver (or other external code) needs to implement the pooling (PooledDataSource). Using this mechanism I won't automatically get the connection pooling provided by JBoss AS, will I?

     

    Also, David Lloyd hinted that a 'regular' (jboss) datasource definition inside an EAR might still be supported (without the ability of an administrator to control it). Can anyone comment on that?

  • 9. Data sources in EAR on AS7?
    Jesper Pedersen Master

    To quote a former colleague "You're doing it wrong".

     

    AS 7 is a completely new architecture compared to the former AS releases. Instead of embedding datasource definitions inside your EAR deployment they should be part of the deployment plan for the application.

     

    As Jason states you can use the @DataSourceDefinition, however you won't get the benefit of being in management controlled environment - and hence you won't get the benefits of the JCA container and all features in the container itself and the JDBC resource adapter, such as prepared statement cache and so on. Look the datasource schema over, and see what you are going to manage yourself plus of course all the management features of the AS itself.

     

    Why not try to use Alexey's CLI tool (jboss-admin.sh) to create the deployment plan that you are looking for - if there are any missing features we are always looking for feedback.

     

    Having said that, this is open source - you are more than welcome to contribute to the project.

  • 10. Re: Data sources in EAR on AS7?
    henk de boer Master

    Jesper Pedersen wrote:

     

    To quote a former colleague "You're doing it wrong".

     

    Jesper, I know the discussion, but IMO it's largely based on opinions, just like e.g. some people prefer slim entities (containing only data), while others prefer entities that do their own persistence.

     

    The classic Java EE approach is to have only references to resources in your EAR, and let the environment to which this EAR is deployed provide definitions for them. This seems to assume a division between developers and operations, where an operations team administrates these resources. No doubt this approach works for some teams, but for other teams this is not the most natural fit.

     

    Most places where I worked don't have this separation, but indeed have people more akin to the role "devops". These are developers who write the code and thus create the resources (datasources, jms destinations) and also configure these resources with the correct parameters for the different stages. Such a developer is also part of the operations team, and thus knows about the addresses of e.g. the database being used in production and the QA stages.

     

    A pure operations team member in this setup does not have intimate knowledge about JBoss AS or about JMS, XA, etc. He or she maintains the servers at e.g. the Linux level, operates the firewall, the switches etc.

     

    So, what happens in this case is that the developer/devops person has to create the various -ds.xml files anyway, but instead of being near the code these now have to be maintained somewhere else. I've seen among others CF-ENGINE being used for this. The problem then is that developers don't really see these files. Since CF-ENGINE also contains tons of other system files (like firewall settings, apache settings, etc) they probably don't have access to this.

     

    Over time, this then runs out of sync. Developers remove datasources from their local config (since the code isn't using it anymore), but this change isn't propagated to the other stages, since the developer simply can't access these. Of course, there can be process put into place that requires developers to write tickets for operations people to make these changes. But since the operations people don't have knowledge about these changes, they merely copy the changes the developer puts in the ticket. Worse, since the developer doesn't see the complete file anymore (he only communicates the changes), he or she doesn't see whether the end result over time is still sane.

     

    In my opinion the EAR-with-resource-references-only assumes a situation where developers are mostly irresponsible cowboys, while operations are sensible, responsible people.

     

    For shops where this is indeed the case, I fully agree that the classic EAR-with-resource-references-only is a sane approach. Huge enterprises are probably the main target audience here. For universal EARs that should be shipped to different clients this approach also makes sense.

     

    However, for smaller and leaner shops with a devops culture where development is for an in-house operated platform, this separation only poses an amount of needless overhead. It's much, much clearer if the EAR contains all configuration for the multiple stages. If for instance a JMS queue needs to be added or removed, this can be trivially done.

     

    To illustrate, our structure is now such:

     

     

    EAR
      conf
        local
          some-ds.xml
          some-hornetq-jms.xml
        dev
          some-ds.xml
          some-hornetq-jms.xml
        live
          some-ds.xml
          some-hornetq-jms.xml
        some-universal-queue-hornetq-jms.xml
      lib
      ...
      META-INF
        jboss-app.xml
    
    

     

    jboss-app.xml is parameterized as follows:

     

    <module>
       <service>conf/${our.staging}/some-ds.xml</service>
    </module>
    
    

     

     

    I've seen this structure, or a structure much like it, at many places. If you search your own forums here you might come across multiple posts where people ask how to include various things in their EAR and stress the importance of being able to do so.

     

    It's not just a structure that we started to use because we don't yet understand how deployments should work. We thought a lot about this and had discussions with various people from various backgrounds (from entry developers to hardened devops, etc) about what would suit their requirements best.

     

    There are simply multiple approaches that work for different situations. An important asset of JBoss AS has always been that it supported both approaches. If you just throw out the support for one of those, because (uhm, because of what reason exactly?), then I really think this does not improve the attractiveness of JBoss AS.

     

    I also wonder how many shops all over the world greatly depend on this feature and will run into major troubles when eventually upgrading to AS 7.

     

    (yes, I guess we could write scripts that unpacks the EAR first and copies the config files from the EAR to the JBoss deploy directory, but this does require such scripts to be able to run on deployment where we now simply do a copy of the EAR and that's it)

     

    Sorry for the long text, but I hope it offers some insight into why packing resources with an EAR can be important to some.

  • 11. Re: Data sources in EAR on AS7?
    henk de boer Master

    Jesper Pedersen wrote:

     

    Why not try to use Alexey's CLI tool (jboss-admin.sh) to create the deployment plan that you are looking for - if there are any missing features we are always looking for feedback.

     

    One use case that I forgot to mention (sorry for making the post even longer), is the local development situation. Currently in our setup developers simply clone a project from the Hg repository and via JBoss tools they deploy it to JBoss AS. This almost can't be simpler.

     

    Requiring developers to create deployment plans before they can run a project doesn't really make things simpler. There's also the issue of configuration for resources being updated in the repository. If those changes don't automatically come in with a pull of the project and automatically deploy, then this really impedes development.

  • 12. Data sources in EAR on AS7?
    Sebastian Koske Newbie

    I agree with Henk,

     

    we use resource definition templates (DS, JMS) in our EARs frequently. This makes sense since their names and most of the configuration details are application specific and maintained by the developers. Plus, we dont want to rely on expert or application-specific knowledge at the customer side where the applications are deployed. So we provide this template and simple skipt-based property replacement (basically for ds-urls).

     

    Yet, we provide our resource definitions as .sar archives (jboss-service.xml). So will this approach still be supported?

  • 13. Re: Data sources in EAR on AS7?
    arjan tijms Novice

    What's the reason that a perfectly fine feature as defining data sources in an EAR is going to be removed? Is this technical or because it's deemed to be a wrong approach to put a data source configuration inside an EAR?

     

    And what about other resources like JMS queues, can they still be defined inside an EAR? Currently our apps define quite a number of queues that are each nothing more than e.g.:

     

    <queue name="aQueue">
       <entry name="/queue/aQueue" />
    </queue>
    

     

    I'm all for separation of concerns where it makes sense, but I see zero value in forcing resources like these JMS queues to be defined externally.

  • 14. Data sources in EAR on AS7?
    Carlo de Wolf Master

    Note that the photoalbum demo also uses this feature.

     

    https://issues.jboss.org/browse/JBAS-8926

1 2 3 4 Previous Next