Some Maven multiproject experiences will be described in this post. Special attention is paid to the dashboard plug-in.
Maven: multi-project and dashboard issues
Generic Maven project structure
Let us first reiterate the canonical structure of a Maven
(sub)project:
This project structure will henceforth implicitly be assumed for
all subprojects constituting an overall superproject.
A canonical multiproject lay-out
Although most if not all plug-ins work flawlessly with the
multiproject structure as outlined below, it is not a universally
applicable lay-out. For example, this lay-out does’t integrate
well with the Eclipse or WSAD IDEs.
However, for a proper understanding, let us first shortly
(re)introduce the canonical multiproject structure, that is
(sometimes implicitly) assumed in most Maven and
Maven plug-in related documentation.
Canonical multiproject structure
The lay-out of a canonical multiproject project is as follows
(where we collapsed the subproject tree structure as was listed
above):
Invoking
$ maven multiproject:site
a target directory is generated containing the maven generated
site.
The advantage is that set aside some customizations and/or
optimizations, a maven.xml
is not needed, hence
your Maven (and Jelly) related knowledge can be restricted
to the bare minimum. Moreover, you project structure lay-out
will be Maven generic!
Below we will focus a bit more on the precise contents of
the project.xml
and the project.properties
,
in order to be able to point out the differences with our
more generalized case that will be discussed thereafter.
Contents project.xml
An example project.xml
has been listed below.
We made use of the possibility to inherit (extend) from a top-level
project.xml
file, located in one of the subprojects.
This is especially usefull when dealing with a project consisting
of a lot of subprojects and is almost always employed.
This project.xml
is responsible for the main (overview)
project site. The dashboard and multiproject plug-ins are configured
here, the former generating a summary of project metrics
(which metrics are to be included can be configured, see
the project.properties
below),
the latter analyzing the version convergency of the
dependencies of all subprojects on (external) artifacts,
such as JAR and ZIP files.
Of course, other plug-ins, such as the mutlichanges plug-in can
and may be configured here as well, but we will refrain from
doing so for the sake of clarity.
A template project.xml
is shown below:
<?xml version="1.0" encoding="UTF-8"?> <project> <extend>sub-project-m/top-level-project.xml</extend> <pomVersion>3</pomVersion> <id>site</id> <name>Overall Site</name> <reports> <report>maven-dashboard-plugin</report> <report>maven-multiproject-plugin</report> </reports> </project>
Contents project.properties
Below we have only listed those properties
from the project.properties
file that are relevant
to the multi-project plug-in (and reactor) and the
dashboard plug-in and associated overall report generation:
#
# Multiproject
#
maven.mutliproject.basedir=.
maven.multiproject.includes=sub-project-*/project.xml
maven.multiproject.excludes=sub-project-DontBuildMe
#
# Dashboard properties
#
# Tells the dashboard plugin not to run its own reactor.
# We’ll use the one run by the multiproject.
maven.dashboard.runreactor = false
#
# Tells the multiproject plugin to execute the dashboard:report-single
# goal on each project when you run it through multiproject:site.
maven.multiproject.site.goals=site, dashboard:report-single
#
maven.dashboard.aggregators = csfiles,cswarnings,cserrors,junittests,junitfailures,juniterrors,junitpassrate,pmdfiles,pmdviolations,simiantdl,jcoveragelipc,jcoverageloc
The multiproject related properties don’t need to be explained,
except maybe for the maven.multiproject.excludes
: this
property is optional and can be used for those subprojects that must
(temporarily) be exempted from the build process.
Dashborad report generation
It turns out that the dashboard plug-in seamlessly integrates
with this canonical multiproject structure. An integrated
summary will be available under the project report section.
An example of such a summary is listed on the Dashboard plug-in page,
http://maven.apache.org/reference/plugins/dashboard/.
Of course, for those subprojects that have dashboard reporting
enabled in the report section of their POM (project.xml
),
a dashboard report for the subproject itself is included in the
project reports section.
Multiproject with root subproject
The need for a more generalized lay-out has been motivated at the
beginning of this document. In practice, this "root"
subproject may be named differently, such as main project, site
project or deployment project.
Multiproject with root dir structure
The lay-out of a canonical multiproject project that contains
a root project on the same level as the subprojects is depicted
below:
The differences with the canonical case are immediately obvious:
- The
project.xml
and
project.properties
have moved to the root
subproject - A
maven.xml
is mandatory now - The root project is on the same level as the subprojects
- Site pages reside in the
site
dir.
All these small differences imply many small intricate mutations
as compared to the canonical case. These changes will be
elaborated on below.
Contents maven.xml
Let us first focus on the maven.xml
file.
<goal name="myProject:site" description="Build site for the master project and all subprojects"> <echo message="+-----------------------------------------------+"/> <echo message="| First building sites of separate sub projects |"/> <echo message="+-----------------------------------------------+"/> <maven:reactor basedir="${basedir}/.." includes="${maven.multiproject.includes}" excludes="${maven.multiproject.excludes}" goals="dashboard:report-single, dashboard:report, xdoc" banner="Build site for subproject " postProcessing="true" ignoreFailures="true"/> <echo message="+----------------------------------+"/> <echo message="| Performing dashboard aggragation |"/> <echo message="+----------------------------------+"/> <attainGoal name="dashboard:aggregate"/> <attainGoal name="myProject:mainSite"/> </goal> <goal name="myProject:mainSite" description="Build main (root/overview) pages"> <echo message="+-------------------------------------+"/> <echo message="| Building main (root/overview) pages |"/> <echo message="+-------------------------------------+"/> <maven:reactor basedir="${basedir}" includes="site/project.xml" goals="allSite" banner="Build site for myProject using " postProcessing="false" ignoreFailures="true"/> <echo message="+-----------------------------------+"/> <echo message="| Copying all to target destination |"/> <echo message="+-----------------------------------+"/> <copy todir="${basedir}/target"> <fileset dir="${basedir}/site/target"> <include name="**/*" /> </fileset> </copy> </goal>
The maven.xml
is composed as follows:
- The
myProject:site
builds the whole
site, i.e. the main site and of all subprojects. - It starts off by invocation of the reactor (standard)
for the site generation of all subprojects. - Note the goals that are invoked for each separate subproject!
This is essential to get the dashboard reports
right. - Thereafter, an aggregate dashboard report is generated.
- The overall site is generated. This will require a
section by itself. Note how the generation of the
main site is "handed over" to the
allSite
goal of themaven.xml
in thesite
directory. - All files are copied to the
target
directory in the root level project directory.
Needless to say, any destination will do, as well as a
deploy to web server, of course.
Contents project.properties
Below we have only listed those properties
from the project.properties
file that are relevant
to the project lay-out (and dashboard report generation):
#
# Multiproject
#
maven.multiproject.basedir=..
maven.multiproject.includes=sub-project-*/project.xml
maven.multiproject.excludes=sub-project-root/project.xml
#
# Dashboard properties
#
# Tells the dashboard plugin not to run its own reactor.
# We’ll use the one run by the multiproject.
maven.dashboard.runreactor = false
#
maven.dashboard.aggregators = csfiles,cswarnings,cserrors,junittests,junitfailures,juniterrors,junitpassrate,pmdfiles,pmdviolations,simiantdl,jcoveragelipc,jcoverageloc,javancssncsstotal,javancssjavadocstotal
The properties and their values should be obvious from inspection.
Notice the dashboard relevant properties especially: not more, not
less is needed.
Finally, for dashboard to work properly, you must
incorporate the following lines in the
project.properties
of each of the
subprojects:
maven.dashboard.rungoals=true maven.dashboard.includes=project.xml
Overall site generation
The site pages, as well as the overview of all the subprojects
and the aggregate dashboard report are generated based on
the material residing in the site
directory. Rationale: since we deviated from the canonical
project lay-out, the multiproject nor the dashboard plug-ins
"know" how to handle this situation, and special
adaptations have been incorporated to cope with this new situation.
Let us first focus on the overview of all subprojects.
This page is generated from the projects-overview.xml
file in the xdocs-templates
directory:
<?xml version="1.0"?> <document> <properties> <title>Overview of projects</title> </properties> <body> <section name="Overview of projects"> <table> #foreach ($reactorProject in $reactorProjects) <tr> <td><b><a href="${reactorProject.file.parentFile.name}/index.html">${reactorProject.artifactId}</a></b></td> <td>${reactorProject.shortDescription}</td> </tr> #end </table> </section> </body> </document>
Using velocity, this template is transformed into a
projects-overview.xml
in the xdocs
directory, which in turn is transformed into an HTML document
by the xdoc
plug-in.
This transformation process is controlled by the
maven.xml
in the site
directory:
<project default="allSite" xmlns:j="jelly:core" xmlns:maven="jelly:maven" xmlns:x="jelly:xml" xmlns:u="jelly:util" xmlns:a="jelly:ant" xmlns:velocity="jelly:org.apache.commons.jelly.tags.velocity.VelocityTagLibrary"> <goal name="allSite"> <attainGoal name="subSite"/> <attainGoal name="dashboard:xdoc"/> <attainGoal name="site"/> </goal> <goal name="subSite" description="Build the overview for all subprojects" > <maven:reactor basedir="${basedir}/../.." includes="${maven.multiproject.includes}" excludes="${maven.multiproject.excludes}" goals="pom" sort="true" banner="Building subprojects overview" postProcessing="true" ignoreFailures="false"/> <attainGoal name="dashboard:aggregate"/> <!-- | | At this point because 'postProcessing' was turned on we have all | the processed POMs available to us in the ${reactorProjects} variable. | | The overall project page can now be generated. | --> <fileScanner var="templates"> <fileset dir="${basedir}/xdocs-templates"> <include name="*.xml"/> </fileset> </fileScanner> <j:set var="templatesX" value="${templates}X"/> <j:if test="${!templatesX.equals('X')}"> <j:forEach var="template" items="${templates.iterator()}"> <j:set var="templateName" value="${template.name}"/> <echo>Processing ${templateName} template ...</echo> <velocity:merge name="${basedir}/xdocs/${templateName}" basedir="${basedir}/xdocs-templates" template="${templateName}"/> </j:forEach> </j:if> <!-- | | The sites of the various subprojects are now copied over to the main | site. | --> <j:forEach var="reactorProject" items="${reactorProjects}"> <echo>Copying target dirs from project ${reactorProject.artifactId} to ${basedir}</echo> <copy todir="${basedir}/target/docs/${reactorProject.artifactId}"> <fileset dir="${basedir}/../../${reactorProject.artifactId}/target/docs"> <include name="**/*" /> </fileset> </copy> </j:forEach> </goal> </project>
Note how
- The reactor merely builds the POM, in order to have
all subprojects available, so that we can iterate over
them. With this iteration, the projects overview page
and the dashboard report can be constructed. - The subSite goal generates the overview page
- Where and how the dashboard aggreation takes place
- The dashboard aggregate data (in
dashboard-data.xml
) is transformed into
a report (indashboard-report.xml
), which
will eventually be transformed into a
dashboard-report.html
. - How the site material of the subprojects is copied over
to the overall destination.
In order for the generation of the overview page to
work properly, the following dependencies have to be
incorporated in the project.xml
in the
site
directory:
<?xml version="1.0" encoding="UTF-8"?> <project> <extend>../bibtexml-top-level-project.xml</extend> <pomVersion>3</pomVersion> <id>site</id> <name>BibTeXML Site</name> <dependencies> <dependency> <id>commons-jelly+tags-velocity</id> <version>SNAPSHOT</version> </dependency> <dependency> <id>velocity</id> <version>1.3</version> </dependency> </dependencies> <reports> <report>maven-multiproject-plugin</report> <!--report>maven-multichanges-plugin</report--> <!--report>maven-tasklist-plugin</report--> </reports> </project>
Moreover, if you want the multiproject convergency report,
you have to enable it here. Do not enable
the dashboard report here: it won’t work, and what’s even
worse, it will overwrite our so carefully aggregated
report with an empty one!
There is a typo at:
#
# Multiproject
#
maven.mu”tli”project.basedir=.
by the way, when i try to keep the master/root project.xml in a dir no subproject is constructed in the end 🙁
The restructuring of our project is completed. However, we decided not to continue (temporarily) with Maven 1.1-beta. Some reasons were specific for our project, while others are more generic:
All this made us decide to wait for the official release.
Some positive remarks to conclude though: Maven 1.1 has a much lower memory footprint indeed, and seems to be much faster, a typical site generation for our project went down from 45 to 30 minutes (approx.).
In fact, we cannot use the Maven reactor right now, because of “out of memory” problems, whereas Maven 1.1 did fine, except for the Dashboard and Velocity template generation.
Currently we are migrating to Maven 1.1, while at the same time restructuring our project. I’ll collect our experiences and describe them in a forthcoming follow-up. I’ll leave a comment with a link to this new post here.
Thank you Rupert for your suggestion as it helps me make my multiproject dashboard work.
I have made a my own improvements though, I think it is better to separate the “multiproject/project.xml” from the “base/project.xml” where all subcomponents inherited. This way you don’t have to edit each subprojects dashboard settings
base/project.properties
–>maven.dashboard.basedir=${pom.getArtifactId()} (this value depends on where you’ve put the base/project.xml file)
I had to add this line to make the multiproject dashboard work:
–>maven.dashboard.includes=${maven.multiproject.includes}
I still did not have time to check out your solution in details, but it looks promising!
The reason I had my project structure laid out as described above was that I started working on a real-life instance of such a project the first of January, so the structure was already completely fixed when I started. I then had to come up with a Dashboard solution for the existing project structure.
But again, your solution looks much neater and should be the to be preferred configuration for new projects.
Thanks for your feedback.
And my structure is actually:
I forgot something else. To each sub projects project.properties you need to add (for project_n):
# Tell dashboard to run in on this project only when run in this directory. This relative directory trick will work when called from the
# main build directory or from this directory.
maven.dashboard.basedir=../project_n
maven.dashboard.includes=project.xml
Otherwise you get the full multi-line dashboard report in each sub project because in the main project.properties you told dashboard to run in the parent directory and include all sub-projects and these properties are inherited by all the sub-projects. So you need to override this in each sub-project to create the dashboard just for it when dashboard is run in its individual directory or called from the main script.
Sorry for posting all this in bits in pieces. I should have planned my post out a little better. Hope its still usefull.
Like this:
basedir
|
|- project_1
| |
| |-project.xml
| |-project.properties
| |-maven.xml
|
|- …
|
|- mavenbuild
| |
| |- project.xml
| |- project.properties
| |- maven.xml
My diagram of teh project structure hasn’t come out right. The mavenbuild directory contains a project.xml, project.properties and maven.xml. They are not in the parent directory as the diagram has come out.
Zeger, Thanks for your post. It was helpful in pointing me in the right direction. I have a project structure with the master build project.xml in one of the sub-project as described in the root case above. However, I think the way you are running the dashboard is overly complicated and also inneficient. I have managed to get it working in a far simpler way with such a project structure.
My structure is:
basedir
|
|- project_1
| |
| |-project.xml
| |-project.properties
| |-maven.xml
|
|- …
|
|- mavenbuild
|
|- project.xml
|- project.properties
|- maven.xml
The projects in project_1, … all inherit from the master script in mavenbuild.
I have my reports set up in the mavenbuild/project.xml:
In the mavenbuild/project.properties I set up multiproject and the dashboard. I also set up clover to output the raw .xml report so that the dashboard can re-use it:
# Set up the dashboard to run in the parent directory and to ignore this project.
maven.dashboard.basedir=..
maven.dashboard.excludes=mavenbuild/project.xml
maven.dashboard.rungoals=false
maven.dashboard.aggregators=csall,junitpassrate,junittests,juniterrors,junitfailures,clovertpc,cloverncloc
# Tell multiproject to work in this directory and to ignore this project.
maven.multiproject.basedir=..
maven.multiproject.excludes=mavenbuild/project.xml
# Tell clover to output the raw .xml report so taht the dashboard can re-use it without running clover again.
maven.clover.report.xml=true
Then in the mavenbuild directory I run the multiproject:site goal and it correctly generates the dashboard for all projects. This is much simpler than what you describe above. It is also more efficient because I have been able to use the rungoals=false setting to re-use the output of the other reports that I am generating. If this were set to true as in your example it would run all my tests, clover, checkstyle and so on two more times! Once when generating the single line dashboard for each individual sub-project and once when generating the full multi-line dashboard for the multi-project parent site.
My project layout is a little different to what you have described here as I don’t have a sub-project within my mavenbuild directory for generating the site. Possibly, I have not fully understood your reasons for doing it that way?
It seems to me that the important trick with my layout is telling the dashboard to work in the parent directory and to exclude the mavenbuild directory with the settings in the project.properties file.
I hope you find this helpful.
I search for blog like this long time.You website is very good!I will come next time!
カジノ
Well done, Zeger! Good to see you’re back.
By the way, I forgot to mention that I got an excellent introduction into Maven myself from my colleague Marcel de Koster. Consequently, this example (code and structure) is also partly based on his work.
As a matter of fact, I did so for the concrete instantiation of the "template" project presented in this post.
Basically, you set up the project structure exactly as discussed in this post, that is, with the root directory on the same level as your subprojects.
You have to play a bit with the project settings in Eclipse, to get the source and conf directories in your classpath, but then everything integrates seamlessly.
You may want to use the mevenide eclipse plugin to keep the dependencies section of your
project.xml
synchronized with your Eclipse classpath.If this isn’t enough to get you started, just leave another comment, and I’ll seriously consider a follow-up!
Great writeup. Could you give some hints how you would align the structure if Eclipse/WSAD is your primary environment?
🙂 stw