10 Replies Latest reply: Aug 27, 2008 7:23 AM by Emmanuel Bernard RSS

Improvements

Emmanuel Bernard Master

Here is a list of issues we encounter when trying to work with the whole maven enchilada.

1. There is no staging server / level when publishing a version in Maven. On a modular project portfolio (say Hibernate), it means you need to push a version in the wild, give it a try with related bits of the project and potentially fix issues that came up during the release process.
In a pure Maven way, thou shalt not republish the same version. The reason comes from:
- the absence of a staging server
- a rather stupid caching approach

Today, people either have to
a. create a new release (which is very long as you need to update doc and a few other things) => not practical
b. override the broken fresh release. People that decided to go ahead of time and try from maven before the official announcement now have a wrong cache

While this can be worked around somehow when a single person is releasing all the dependent modules, there is no solution today when the release is done by multiple people.

2. There is no (that I am aware of) continuous build integration of JBoss AS using snapshots, so errors come up late in the game when final releases are already in Maven. This makes problem 1. even worse.

  • 1. Re: Improvements
    Andrew Rubinger Master

    The problem with snapshots, and the reason for their ban in AS component-matrix, is because a version of X.Y.Z-SNAPSHOT is not enough to uniquely identify a version to make a build reproducible:

    * By default, Maven local repo is reconciled only once/day to check for newer versions.
    * The notion of "newest" snapshot is defined by a timestamp, a property of the *publishing machine*. Rouge system clocks wreck havoc on all.
    * A developer may have installed a newer version locally than is available to all on snapshots.jboss.org

    However, take a look at a typical directory:

    http://snapshots.jboss.org/maven2/org/jboss/ejb3/jboss-ejb3-proxy/0.1.2-SNAPSHOT/

    I'd like to test out explicitly specifying an extension in the version, ie:

    <version>0.1.2-20080818.031348-4</version>


    If Maven handles this case we may again put snaps up for integration testing in AS (using this format only), and hold off preparing a proper release until it's confirmed that the version we're testing is valid.

    S,
    ALR

  • 2. Re: Improvements
    Carlo de Wolf Master

    You can only do continuous integration if the snapshot you're depending on doesn't have transitive dependencies which are snapshots themselves.

    For example AS -> ejb3-core SNAPSHOT -> ejb3-interceptors 0.1.0-SNAPSHOT -> ejb3-metadata 0.1.0-SNAPSHOT.

    If I were to deploy a new ejb3-metadata snapshot, which invalidates the ejb3-interceptors test suit, then AS is screwed.

    It should have been:ejb3-core SNAPSHOT -> ejb3-interceptors 0.1.0-20080821.010101-1 -> ejb3-metadata 0.1.0-20080820.010101-1.

    Now if I deploy a new ejb3-metadata it should automatically be picked up by interceptors which deploys after passing the test suite (or fails horrible). In both cases AS stays on the correct ejb3-metadata.

    We've discussed this in the past resulting in: http://jira.codehaus.org/browse/MDEPLOY-77.

  • 3. Re: Improvements
    Emmanuel Bernard Master

    well I was more thinking about an extreme version of that. A extreme CI.
    You take all snapshots of all projects all the time. So you see shit coming and you see when you break 95% of the app server.

  • 4. Re: Improvements
    Andrew Rubinger Master

    Ah crap, I hadn't considered snapshots brought in as transitive dependencies.

    MDEPLOY-77 would require that all transitive dependencies also be deployed (as a locally-installed snap is not available to everyone else).

    Bottom line: we won't come to a happy resolution without cracking into Codehaus and making the enhancements we want.

    Also, I see the following problems with "taking all snapshots of all projects all the time":

    * New development in a particular project may (knowingly) break backwards-compatibility, and is not intended for integration with AS.
    * IT resources are limited and snapshots are cleared from the repo after 30 days
    * AS should be choosing which versions of which components it wants, not the other way around (ie. projects imposing its latest snap upon AS).

    S,
    ALR

  • 5. Re: Improvements
    Emmanuel Bernard Master

    I partly agree with you Andrew.

    I think the right level for a CI / Snapshot use is to define the targeted branch (say 3.3 for Hibernate Core) at the Jboss AS level. From here, the snapshot build of JBoss AS should get the latest published snapshot for that release (ideally it should get it from SVN, I don't see why we should have to manually push a snapshot out).

    Now, when it's time to release AS (beta, cr, ga, alpha etc), specific versions (ie non snapshots) should be chosen manually (as it is today), no question about that.

  • 6. Re: Improvements
    Paul Gier Master

     

    "epbernard" wrote:
    well I was more thinking about an extreme version of that. A extreme CI.
    You take all snapshots of all projects all the time. So you see shit coming and you see when you break 95% of the app server.

    I just wanted to note that this is how maven is set up normally. It will by default check daily for updates of snapshot dependencies. And how the AS build was working until Dimitrius started locking down versions to do a release. One of the problems is that in order to do a beta or RC release of the app server the versions of the components have to be locked down otherwise you have a build that is not reproducible.

  • 7. Re: Improvements
    Andrew Rubinger Master

    I'm looking into what it'll take to restrict a Maven deployment to snapshots that are available remotely, and with static version numbers.

    "epbernard" wrote:
    I think the right level for a CI / Snapshot use is to define the targeted branch (say 3.3 for Hibernate Core) at the Jboss AS level. From here, the snapshot build of JBoss AS should get the latest published snapshot for that release (ideally it should get it from SVN, I don't see why we should have to manually push a snapshot out).


    Interesting, presently we have no mapping between SCM location and libs.

    So the logic could become:

    1) I depend on http://anonsvn.jboss.org/whatever/
    2) If SNAPSHOT location, checkout and build from source. If a release, use a cached lib available in repository.jboss.org

    A brief look into Maven sources reveals that this kind of thing strikes at the heart of a largely unpluggable/unmodular dependency resolution core, so this isn't as trivial as extending/enhancing a plugin.

    For instance, the Maven Dependency Plugin is really a hook, it doesn't actually perform any resolution as the name implies.

    S,
    ALR

  • 8. Re: Improvements
    Carlo de Wolf Master

    So is the release manager / release plugin. The plugin is just a simple facade around the core release manager.

    If we ever want to get well-defined snapshots they must come from the release manager.

  • 9. Re: Improvements
    Andrew Rubinger Master

     

    "wolfc" wrote:
    If we ever want to get well-defined snapshots they must come from the release manager.


    Why? It's maven-deploy-plugin which "Uploads the project artifacts to the internal remote repository". Far as I can see, we just need a mechanism to restrict a snapshot deployment to one with explicit snapshot dependencies, not the generic X.Y.Z-SNAPSHOT format.

    S,
    ALR

  • 10. Re: Improvements
    Emmanuel Bernard Master

     

    "pgier" wrote:
    "epbernard" wrote:
    well I was more thinking about an extreme version of that. A extreme CI.
    You take all snapshots of all projects all the time. So you see shit coming and you see when you break 95% of the app server.

    I just wanted to note that this is how maven is set up normally. It will by default check daily for updates of snapshot dependencies. And how the AS build was working until Dimitrius started locking down versions to do a release. One of the problems is that in order to do a beta or RC release of the app server the versions of the components have to be locked down otherwise you have a build that is not reproducible.


    Dimitris position totally makes sense. We need a parallel run where snapshots of X.Y libs is used to build the edge version. from those results, dimitris can pick and chose who is stable and ask for a release of particular libraries.