12 Replies Latest reply on May 25, 2010 4:56 PM by pgier

Quicker AS build?

jaikiran Jan 28, 2010 4:01 AM

When building the AS trunk, i usually see numerous snapshots being downloded. This seems to happen (atleast) once a day. For example, this morning when i triggered a build on the "testsuite" module of AS trunk:

..testsuite$] ./build.sh clean jars

I see a whole lot of snapshots being downloaded:

init-dependencies:
[maven:dependencies] [INFO] snapshot org.jboss.jbossas:jboss-as-aspects:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[maven:dependencies] Downloading: org/jboss/jbossas/jboss-as-aspects/6.0.0-SNAPSHOT/jboss-as-aspects-6.0.0-20100128.002825-636.pom from repository snapshots.jboss.org at http://snapshots.jboss.org/maven2
[maven:dependencies] Transferring 5K from snapshots.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop/2.1.8.GA/jboss-aop-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 20K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop-parent/2.1.8.GA/jboss-aop-parent-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 18K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop-aspects/2.1.8.GA/jboss-aop-aspects-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 3K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop-asintegration-core/2.1.8.GA/jboss-aop-asintegration-core-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 5K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/pluggable-instrumentor/2.1.8.GA/pluggable-instrumentor-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 1K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop-asintegration-jmx/2.1.8.GA/jboss-aop-asintegration-jmx-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 6K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop-asintegration-mc/2.1.8.GA/jboss-aop-asintegration-mc-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
....

This snapshot download lasts anywhere between 15-20 minutes (and sometimes even half an hour) on my system. Is there any way this can be avoided?

By the way, my understanding was that the AS doesn't depend on any snapshot dependencies. So why are these being downloaded in first place?

1. Re: Quicker AS build?

dimitris Jan 28, 2010 4:17 AM (in response to jaikiran)

I've asked Paul the same thing:

"It's an annoying bug [1] in Maven where it will check for snapshot updates according to the update policy (once per day by default) even if an up to date copy is installed locally. The good news is that it still uses the correct jars in the build, even though it looks at the remote repo, and I think it is already fixed in Maven 3. But we're stuck with it until Maven 3 goes to GA.

[1]http://jira.codehaus.org/browse/MNG-4326"
Actions
2. Re: Quicker AS build?

jaikiran Jan 28, 2010 4:19 AM (in response to jaikiran)
BUILD SUCCESSFUL Total time: 26 minutes 11 seconds

That's the total time on the build of "testsuite" module. Out of which 24 minutes were spent in downloading the snapshots.
Actions
3. Re: Quicker AS build?

jaikiran Jan 28, 2010 4:34 AM (in response to dimitris)

dimitris@jboss.org wrote:

I've asked Paul the same thing:

"It's an annoying bug [1] in Maven where it will check for snapshot updates according to the update policy (once per day by default) even if an up to date copy is installed locally.

[1]http://jira.codehaus.org/browse/MNG-4326"
Hmm, maybe as a workaround, we need some kind of custom "never update" policy as the default for the AS build.
Actions
4. Re: Quicker AS build?

wolfc Jan 28, 2010 5:33 AM (in response to jaikiran)

You both are missing the real issue, it's downloading a snapshot of a dependency that really is a module within AS.

Instead of downloading such a snapshot, it should build the other module.
Actions
5. Re: Quicker AS build?

jaikiran Jan 28, 2010 6:04 AM (in response to wolfc)

wolfc wrote:

You both are missing the real issue, it's downloading a snapshot of a dependency that really is a module within AS.

Instead of downloading such a snapshot, it should build the other module.
Yep, and that bug has been acknowledged in http://jira.codehaus.org/browse/MNG-4326

And it doesn't have a fix in Maven-2.x
Actions
6. Re: Quicker AS build?

pgier Jan 28, 2010 9:58 AM (in response to jaikiran)

jaikiran wrote:

dimitris@jboss.org wrote:

I've asked Paul the same thing:

"It's an annoying bug [1] in Maven where it will check for snapshot updates according to the update policy (once per day by default) even if an up to date copy is installed locally.

[1]http://jira.codehaus.org/browse/MNG-4326"
Hmm, maybe as a workaround, we need some kind of custom "never update" policy as the default for the AS build.
You can set the update policy for the snapshots repo to "never" using your settings.xml.
http://maven.apache.org/settings.html#Repositories

Then when you want to update the snapshot dependencies, use "mvn -U install"
Actions
7. Re: Quicker AS build?

brian.stansberry Jan 28, 2010 10:16 AM (in response to pgier)

Thanks, Paul, that seems like excellent advice.

I know 99.9% of the time my interest in snapshots is because I'm working on two projects simultaneously and want the second to see the results of the first. I only want snapshots from my local repo.

The other .1% of the time someone has published a snapshot I need and told me about it, in which case I should use -U.

0% of the time am I interested in some random snapshot on snapshots.jboss.org.
Actions

8. Re: Quicker AS build?

jaikiran Jan 29, 2010 1:57 AM (in response to pgier)

Gave that a try this morning:

<repository>
    <id>snapshots.jboss.org</id>
    <url>http://snapshots.jboss.org/maven2</url>
    <releases>
    <enabled>false</enabled>
    </releases>
    <snapshots>
    <enabled>true</enabled>
    <!-- We don't want to download snapshots by default
        http://community.jboss.org/thread/147383 -->
    <updatePolicy>never</updatePolicy>
    </snapshots>
</repository>

No more downloads from the snapshot repository during the build. The build is definitely faster now.

It however, still "checks" the snapshot repo though:

init-dependencies:
[INFO] snapshot org.jboss.jbossas:jboss-as-aspects:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-system-jmx:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-system:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-cluster:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-server:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-deployment:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-security:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-connector:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-console:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-management:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
...

Doesn't download anything (which is good news).

9. Re: Quicker AS build?

dimitris Jan 29, 2010 4:14 AM (in response to jaikiran)

All right, that worked great for me too:
.....
<updatePolicy>never</updatePolicy>

I'm wondering, can we have those repository settings applied by default as part of the AS checkout ? I.e. for someone that just checks out the code and wants to start coding, if he doesn't have overrides in his local settings.xml, sensible defaults should apply.
Actions
10. Re: Quicker AS build?

ropalka Jan 29, 2010 8:48 AM (in response to jaikiran)

Thanks for the advice Paul, really useful.
Actions
11. Re: Quicker AS build?

whitingjr May 5, 2010 6:44 AM (in response to ropalka)

An alternative technique is to pass to maven the offline flag which avoids artifact downloading

> mvn --offline compile
or
> mvn -o compile
Actions
12. Re: Quicker AS build?

pgier May 25, 2010 4:56 PM (in response to whitingjr)

Just to provide some more information on this, the problem is that with the current AS build, the Maven Ant tasks have to re-resolve the dependencies during the dist build. This is because the antrun plugin by itself does not provide enough information to the Ant part of the build. I'm currently working on a release of the maven-antrun-plugin that will allow me to remove the Maven Ant tasks from the dist part of the build. This means that the dependencies will only have to be resolved once, and you won't see snapshots being downloaded if they were just installed.

Also related to this is an upstream issue where Maven will check (but not download) the snapshot repositories even when there is an up to date version available in the local repo. This has been fixed in Maven 3.
http://jira.codehaus.org/browse/MNG-4326
Actions

Go to original post