Java 5.0 (Tiger) – MetaData and Annotations – Introduction and thoughts on application

I have given the meta-data features in the J2SE 5.0 Tiger Release some more thought. Reading – in this order – through the following articles, I rather quickly had the feeling that I understood what it is all about:

• Annotations in Tiger, Part 1: Add metadata to Java code, How to use Java 5’s built-in by Brett McLaughlin (brett@newInstance.com).
• Annotations in Tiger, Part 2: Custom annotations, Write your own annotations in Java 5 by Brett McLaughlin (brett@newInstance.com).
• Taming Tiger, Part 3 Decorate your code with Java annotations, by Tarak Modi

Especially the third article was a big help in understanding what use is the custom meta-data.

Basically, meta-data in Tiger allows us to specify things about our code through Annotations- more specifically about Classes, Methods, Interfaces, Member Variables , Method Parameter, Package, Constructor and (!) Annotation Type. Annotations look like this:

@com.oreilly.tiger.ch06.InProgress
@TODO("Figure out the amount of interest per month")
public void calculateInterest(float amount, float rate) {
  // Need to finish this method later
}

They use the @ character and are included just prior to the ‘thing’ they describe (Method, Class etc.) In the example from Brett McLaughlin’s article, we specify through annotations that a particular method has the status ‘InProgress’. We also annotate that method as having work to undergo: “Figure out the amount of interest per month”. Through annotations, the source code becomes the meta-data repository: data about the code segments is stored within the code.

Annotations are very much like the indicators we already use in JavaDoc comments, such as @author, @version etc. There is a big difference though: The JavaDoc ‘annotations’ are only recognized by the JavaDoc-let. They have no meaning to -most- IDEs and play no role in the runtime environment. Tiger’s annotations, at least potentially, have a much wider scope:

Annotations can be picked up by IDEs for reporting, navigating, analysis and qa purposes. This could be very much like the way the Eclipse IDE picks up on //TODO code comments: Eclipse presents a Task List based on these TODO ‘annotations’. However, //TODO is gimmick in Eclipse, not a generic Java J2SE feature. With Tiger it seems likely that similar annotations are proposed by various IDEs and at some point some level of standardization will be reached. I bet TODO (or NeedsWork or something similar) will become a more or less standard annotation, and the Java IDEs will be able to produce task-lists based on that annotation. At the same time, IDEs can use status annotations, such as InProgress, to create a ‘project dashboard’: how many classes and methods are Done, how many are in progress, how many tasks are still open etc. You could even extend on this and include in the annotation the estimated work to be done, the required resources, the developer who has been assigned the task etc. IDEs could read and report that information and also provide GUI support to set such values.

Annotations can be processed into the documentation JavaDoc has made use of a limited set of annotations for quite some time: @deprecated, @author, @version, @see, @throws etc. With the Annotations in J2SE 5.0, you can specify that annotations are to be processed into the documentation. That means: if you want that to be the case – and you can configure that per annotation type – you can have the annotation included in the documentation. Note: you can only specify that you want the annotation in the documentation, then you depend on the JavaDoc generator to act on that ‘hint’.

Annotations can be picked up by the Compiler – Use of annotations need not be confined to design-time. During compilation, the compiler can pick up annotations. These annotations can indicate for example that a method is deprecated. The compiler can report warnings for classes that make use of such deprecated classes. Another type of annotation that – eventually – could be picked up by compilers is an narrowing of the type of parameters or method-return values; even though a parameter is defined as Collection or even Object, it may very well be that in actual fact there is a more detailed requirement from the types or values passed into the method. If such requirements are specified in structured annotations, the IDE or the compiler could act on them.

Currently, Tiger supports three standard annotation types that are recognized by the Java Compiler: Override, Deprecated and SuppressWarnings
From Brett McLaughlin’s article:

Override should be used only on methods (not on classes, package declarations, or other constructs). It indicates that the annotated method is overriding a method in a superclass. Like Override, Deprecated is a marker annotation. As you might expect, you use Deprecated to annotate a method that shouldn’t be used anymore. Unlike Override, Deprecated should be placed on the same line as the method being deprecated (why? I’m honestly not sure),.
You shouldn’t have any trouble figuring out what this one does, but it’s not always obvious why this annotation type is so important. It’s actually a side-effect of Tiger’s all-new set of features. For example, consider generics; generics make all sorts of new type-safe operations possible, especially when it comes to Java collections. However, because of generics, the compiler now throws warnings when collections are used without type safety. That’s helpful for code aimed at Tiger, but it makes writing code intended for Java 1.4.x or earlier a real pain. You’ll constantly receive warnings about things that you’re not at all concerned about. How can you get the compiler to leave you in peace?

SupressWarnings comes to the rescue. SupressWarnings, unlike Override and Deprecated, does have a variable — so you use the single-annotation style for working with it. You can supply the variable as an array of values, each of which indicates a specific type of warning to suppress. Take a look at the example in Listing 5, which is some code that normally generates an error in Tiger..

Annotations can be used by automated tools for building, testing, auditing or otherwise processing of Java source code. – Annotations can be used by tools that somehow process Java Source Code. Such tools can both read and write annotations! Annotations can contain the results of quality checks (auditing of standards and rules), hints for generating and/or performing unit-tests, documented exceptions to some otherwise validated guidelines, contain structured documentation data etc. As tools start to make use of Tiger features, they will probably allow use of Annotations to have better control from a more logical location (the source code rather than some external configuration file) over processes performed by the tool.

Annotations can be used at runtime – as extension of Reflections. This means that from your own code you can find out stuff about Classes – very much like Run Time Introspections and Reflection. You can get hold of the annotations that apply to a class, a method, a parameter etc. and for each annotation, you can get the values associated with it. Note that this requires the annotations to have been defined with Retention at RUNTIME; that means to the Compiler: leave annotation in class-file and to the JVM: load annotation and make available through runtime-accessing mechanism. Using annotations at run-time gives you a meta-layer of programming that you can use for example to ensure that you have a compatible set of Class versions working together or which of two equivalent methods you should be calling given the circumstances.

Annotations – design time vs. runtime

When you specify a custom annotation-type, you can indicate whether annotations made for that type should be available only in the source code, also in Java-class code and possibly even at run-time in the JVM. From Brett McLaughlin:

A meta-annotation you want to get under your fingers is Retention. This meta-annotation is related to how the Java compiler treats the annotated annotation type. The compiler has several options:
* Retain the annotation in the compiled class file of the annotated class, and read it when the class first loads
* Retain the annotation in the compiled class file, but ignore it at runtime
* Use the annotation as indicated, but then discard it in the compiled class file

These three options are represented in the java.lang.annotation.RetentionPolicy enum, shown here:

package java.lang.annotation;

public enum RetentionPolicy {
  SOURCE,   // Annotation is discarded by the compiler
  CLASS,    // Annotation is stored in the class file, but ignored by the VM
  RUNTIME   // Annotation is stored in the class file and read by the VM
}

As you should expect by now, the Retention meta-annotation type takes as its single argument one of the enumerated values you see in this listing. You target this meta-annotation to your annotations, as shown below:

@Retention(RetentionPolicy.SOURCE)
public @interface SuppressWarnings {
  // annotation type body
}

As you can tell from here, you can use the shorthand form here, because Retention has a single-member variable. And if you want the retention to be RetentionPolicy.CLASS, you don’t have to do a thing, because that’s the default behavior.

Having access to annotation at runtime, alongside with RTTI/reflection seems a very useful thing to me – and probably the main differentiator with ‘custom tags’ in JavaDoc comments. It allows you to write code that for example checks information about classes and methods you intend to use or call. Such information could for example specify information related to security, performance, resource usage, dependency on external elements, version and revision history, copyright details, etc.

The paper Taming Tiger, Part 3 Decorate your code with Java annotations, by Tarak Modi contains a clear code example of accessing annotation information from your own program code.

Release Monitoring System

Thinking about meta-data like this took me back to memories of many years, when my former colleague Frank Brink of Oracle and myself conceived of a way to guard the installation and patching process of our tool Echo/2000, later called Oracle Echo. This tool consisted of many Oracle PL/SQL packages, tables and views, Oracle Forms modules etc. Since the tool was meant to support Software Configuration Management processes, we thought it extremely important that our own tool’s installation, upgrade and patching went very smoothly. As part of that effort, we created a tool that allowed us -at runtime – to verify whether the installed object-versions were indeed the ones required for the current patch-level. We used a configuration-file that contained a list of all objects that made up the release (tables, Forms, views, packages etc.) and for each object the required version. Our run-time release monitor read this configuration-file and subsequently called upon each component in the application to a) confirm it was there and b) report back its current version label. This version label was then compared to the one set down in the configuration file and any deviations were reported with flashing red fields.

The real key in this setup, which worked quite well actually, was the ability to access each object’s meta-data: its version label. For Oracle database objects such as Tables and Views there is the ability in the database to record meta-data through a COMMENT. A Forms Module provides several meta-data properties that we could use. For PL/SQL packages, things were trickier: no meta-data can be stored about them. The solution we chose was to implement a function GET_VERSION_LABEL() in each package; this function would return a String that contained the actual version label. However, it never felt quite right to implement a function that did not really belong to the application.

With Java Classes – we did not use them very much in those days – you basically had the same situation; no proper way to record meta-data, so you would have to use a special method getVersionLabel() or a STATIC FINAL String. Now the Annotations introduced in Tiger – J2SE 5.0 – finally give us the layer of meta-data we needed: record information about the class, outside the program code proper but inside the Java Source and accessible at runtime!

Now we can create for our Java applications a runtime verification tool that checks whether the correct versions of all Classes are available. It would do so in three steps:

1. Load Class.forName() for all classes in the release definition file (some XML document generated from Stripe/Label/Release definition in Source Code Control system)
2. Get Annotation of type Version and the values associated with its members
3. Verify whether value of label equals the label specified in the release definition file and report any violations.

Another version-related piece of information we could easily record using an annotation: Version Dependencies. What I mean is the following: from the import statements in a Java Class, it is quite clear which classes are used by a certain Class. They give us the dependencies from the class on other classes very rapidly. What they do not tell us though is the specific version dependency. In general, even though we specify import packagex.Classy, we typically do not mean just any old version of that Classy; we really need to have a version that includes that particular method that was only introduced in Java 1.3.1 or that version that still contains that method that was later deprecated. It is often not enough for complete reliable build-information to only have the import statements. I can see value for an annotation that would define the minimum and maximum version of certain classes that a particular class depends upon.

Summary/Conclusions

Annotations are interesting. They will really prove useful once IDEs and other tools start making use of them. I expect to see some sort of standardization on a substantial number of annotations. The current pre-defined set is pretty limited. I would also expect somewhat more support in the compiler for dealing with user defined annotations; the same for JavaDoclets.

The ability to access annotations at runtime is the most fascinating. I do not yet feel like I have grasped all options open to us because of that runtime capability.

Other Resources

Collection of useful links for Java 5.0 in general and annotations in particular
Peeking Inside the Box: Attribute-Oriented Programming with Java 1.5, Part 1 by Don Schwarz(06/30/2004) Pretty advanced examples of using Annotation to drive byte-code instrumentation – using BCEL (from Apache Jakarta), both at Compile Time and Class Load time. Don argues that Source Code instrumentation is not yet supported by a very good tool. XDoclet allows generation of entire classes – but not modifying classes based on Annotations. JavaCC and ANTLR could be options in that direction.

Peeking Inside the Box: Attribute-Oriented Programming with Java 1.5, Part 2 by Don Schwarz (07/21/2004) More advanced stuff on Instrumenting Bytecode during execution for example.

Annotations in J2SE 1.5 (Tiger)—Part 1 Jeff Langr A very interesting example of using annotations to inform JUnit to skip certain methods for unit-testing. Very practical and easy to follow! The second installment – Annotations in J2SE 1.5 (Tiger)—Part 2 -elaborated on the JUnit example and showed how to make use of the member-value pairs that can be associated with Annotations. Again, very useful!

Introduction to SUN’s Annotation Processing Tool (APT) that can process annotated source code: First, apt runs annotation processors that can produce new source code and other files. Next, apt can cause compilation of both original and generated source files, thus easing the development cycle.
Why Should I Use apt?
Many of the intended use cases for annotations involve having annotations in a base file hold information that is used to generate new derived files (source files, class files, deployment descriptors, etc.) that are logically consistent with the base file and its annotations. In other words, instead of manually maintaining consistency among the entire set of files, only the base file would need to be maintained since the derived files are generated. The apt tool is designed for creating the derived files.
Compared to using a doclet to generate the derived files based on annotations, apt

• has a cleaner model of the declarations and current type structure of programs
• uses a more contemporary API design, such as returning generic collections instead of arrays and providing visitors to operate on declarations and types
• supports recursive processing of newly generated files and can automatically cause compilation of original and generated source files