The state of Java [developers] – reflections on Devoxx 2019

I attended Devoxx Belgium – November 2019. The yearly gathering of over 3000 Java developers (numbers provided by Devoxx website). Maybe not all of them Java and perhaps some not even developers. But by and large … Java and software development are the core themes.

This conference has taken the place of JavaOne as the premier venue for the Java community – to exchange ideas, make announcement, promote open source projects and win the hearts and minds of the community at large. It is a great place to learn , get comforted by the pains that others go through such as much as you are yourself, get answers to burning questions and most of all: be inspired. I leave Devoxx with a head full of plans, ideas, intentions, question and ambitions. It will sustain me for a long time. And if I need more – I will check the online videos at YouTube where almost all talks are available.

In this article – I have tried to “persist” some of the ideas and findings that are spinning around in my head. I am aware – and so should you be, my dear reader – that there is a bit of bias involved. The conference offered curated content: decisions were made by the organizers about what subjects to include in the agenda – and which ones not. Perhaps topics that are very relevant were excluded in that way. I also did not visit all sessions: I chose sessions that fit in with my highly personal frame of mind (even though I try to attend some sessions way out of my comfort zone).

Some of my conclusions are not well founded on objective fact and measurements; they reflect my sense of the buzz and general sentiment at this conference – highly influenced by my own preferences and the people I talked with (and not those I did not talk to). With all these caveats, I believe I did capture something that is relevant – at least to me going forward. At the same time I would like to invite you to add comments to this article, to give me a piece of your mind. What did you learn and conclude? Do you concur with what I deduced or do you have an alternative opinion to share?

Shakers and Movers, Hot and Lukewarm and Cool (or not so cool)

Some technologies, tools, frameworks, standards, themes are hot, others are distinctly not. Devoxx is a great place to get feel for what is happening and what is running out of steam. There are several ways of classifying. In the end, I give this ‘gut feel’ based classification.

• Foundational (everyone is using this, no discussion needed) : Maven, Java (8), REST (& JSON), containerized, Kubernetes, JUnit, IntelliJ IDEA, Jenkins
• Strong contenders (close to widespread or even general adoption, could be a relatively new very promising kid on the block ): GraalVM, Kotlin, Quarkus, Micronaut, Visual Studio Code, PostgreSQL, Reactive style, Netty, Microprofile, Go, DevOps, production environment testing (canary, A/B), microservices
• To watch: RESTEasy, Knative, Apache Pulsar, Rust
• Under pressure (apparently losing ground ): Eclipse, Spring, Grails, Scala, Groovy, Reflection & Dynamic Class Loading, Java EE/Jakarta, JBoss/WildFly | WebLogic | WebSphere
• Fading into background: Swing

Themes & Messages

Java runtime – lean and fast

Prepared for Serverless Functions and Containerized Microservices (dynamic, horizontal scalability on container platforms)

There is a very clear trend of being smarter about building applications to enable being better with running them. By removing stuff at compile time that will not be used at runtime anyways, we can create smaller uberjars and get away with trimmed down Java runtime environments. By inspecting code at pre-compile time – much of the work that is typically done at run time (with reflection and dynamic class loading) can be handled by source code manipulatio. This includes weaving in aspect code, manipulating code based on annotations.

Some concrete aspects to this theme: 

• Smart container image building for Java applications (Jib – for quick because smart image rebuild)
• Smart compile time optimizations (Quarkus, Micronaut – for expanding annotations, chucking out unneeded classes, injecting code)
• Native Image (GraalVM, Quarkus, Micronau – create native image/platform specific binary executable with small size, small memory footprint and quick startup)

Do not do expensive runtime stuff such as reflection, dynamic proxies, AOP and dynamic class loading

DevOps – the Developerator

The distinction between development and operations is rapidly becoming meaningless. A separate operations department may be concerned with the platform (Kubernetes and all underlying IaaS). However, application operations are done by the same team that has created and rolled out the software. Testing is increasingly done in Production (with canary workloads), monitoring is being upgraded to provide immediate insight to the Developerators, mean time to repair is reduced through automated build, regression test and (controlled) release. There is no handover to ‘another department’ or even ‘the Ops-people on the team’.

This not a bane for developers. It is actually a boon. To have a rapid feedback cycle from creating code to having that code being used and seeing the metrics of that usage is exhilarating. Being informed of an issue and being able to analyze the issue, develop a fix and release solution all in a matter of hours is equally fulfilling. Okay, having to do that at night is not great. So perhaps do no release major changes just before you go home for the day. Or ever: try to break up changes into smaller changes – perhaps using feature toggles or flags to release code that is not necessarily active.

Clean Code

80-90% of IT budget is spent on maintaining and evolving systems, not on building them from scratch. Code is read 10 times more often than it is written. Even the original author of the code will not have any recollection of the how and why of her own code after a weeks’ spent on other topics. Productivity, quality and joy in the lives of developers is increased with clean code – that is easily read and understood. Code whose meaning is clear. What it does and why it does that.

• Naming of variables, methods and classes: clear naming is mandatory.
• Methods should be short (one page in the IDE – 20 lines of code or preferably less)
• Methods should not have more than three parameters
• Parameters should not be of type Boolean – at least not used as flags to request alternative behaviors of the method
• Methods that return a result should not have side effects; methods that return null can have a side effect; make clear in the name of the method what that side effect is
• Comments in code should rarely be used – the code should speak for itself. However, comments that reveal workaround for non trivial issues and bugs are valuable. Or that explain a special corner case.
• (serious) Peer reviews should ensure that code does speak for itself.

You should only commit code that you are prepared to add to your CV.

Watch: this talk by @VictorRentea on Clean Code: http://youtube.com/watch?v=wY_CUkU1zfw…

Developer – know your IDE! For productivity, refactoring, uniform code [quality], instant testing

Get the most out of your IDE. For productivity and fun. For overcoming fears of refactoring – and to apply refactoring. One of the most important refactoring tools: Extract Method.

For a still increasing number of people, that IDE is IntelliJ. At the same time, there is a meteoric rise in the use of Visual Studio Code. Eclipse is losing ground rapidly. Who even remembers NetBeans or Oracle JDeveloper?

 

Letting go, Learning and Unlearning, Deprecate

The challenge to get rid of stuff – old ways of doing things, old technologies with unnecessary shortcomings, old fears and long held beliefs – is tremendous. On various levels – from psychological to economic. We have to be prepared to unlearn things – even or maybe especially things we have done and known and believed in for many years. At least be prepared to change and embrace new ways of doing things if they are better. With our experience, we should be able to judge whether they are better – if we can be really honest.

Being able to get rid of technologies that are really of yesteryear is a challenge: there are risks involved, there is no immediate business benefit [so how to find budget, time and priority]. However, continuing on with those technologies is risky and in de long run similarly challenging [ unsupported, vulnerable technology for which no developers and admins can be found, that are unproductive and eventually may not run on the platform, the OS or the hardware].

The Java Platform – OpenJDK, distributions, HotSpot and GraalVM

OpenJDK is an open source project – with sources for the open source implementation of the Java Platform Standard Edition. OpenJDK does not ship binaries. Various companies provide builds or binary distributions based on OpenJDK – a bit like various companies provide their own distributions of the Linux (open source) Kernel. Oracle happens to be one of them – but not necessarily still the leading one. Other builds are available from AWS (Corretto), Azul (Zulu), RedHat, SAP and IBM. Rumours are spreading that Microsoft will soon ship its own build as well. Note: as one source told me, up to 75% or more of the commits on the OpenJDK projects are made by Oracle staff. Some of the negative emotions projected at Oracle may be softened a little if people would be aware of that fact. Oracle is still the by far biggest contributor to the evolution of the Java platform.

The HotSpot Virtual Machine is part of OpenJDK. As such, both the C1 and C2 JIT compilers are there. These compilers have been implemented in C/C++. It has become quite hard to further evolve especially the C2 compiler – although that certainly is going on with for example Java language enhancements such as Valhalla, Panama and Loom. All (?) JVM distributions ship the HotSpot compilers.

Oracle Labs produced GraalVM. Under this umbrella project, several components are worked on. One is a new JIT compiler that can replace the current C2 HotSpot compiler. This compiler is created to better optimize modern Java Byte code patterns that are getting more common for example with Java Byte code originating from Scala code or from modern Java features such as Streams. The GraalVM JIT Compiler can be enabled in existing JVM environments to implement the JIT compiler and as such bring modern optimization patterns (this I believe is the approach taken by Twitter to run their Scala applications).

Also read this quite good Baeldung article.

Note: it seemed at Devoxx that 80% of attendees was on Java 8 and 20% was on later versions already.

The Star of the Show: GraalVM

If I would have to decide what the biggest star of this week of Devoxx was, I would say the prize goes to GraalVM. GraalVM started life as a research project in Oracle Labs – to see if a replacement could be created for C2 – the C++ based HotSpot JIT compiler that had gotten very hard to maintain and optimize for modern code paths. In seven years, GraalVM has expanded quite a bit. It has delivered the JIT compiler that we can now all plug into our JDK environments to improve [in certain cases substantially] the performance of our Java applications. Additionally, GraalVM can run applications written in other – non JVM – languages such as JavaScript/Node, R, Ruby, Python and LLVM languages (C/C++, Rust, Swift, ..) and it can run hybrid or polyglot applications that combine multiple languages.

A feature of GraalVM that played an important role during Devoxx this year is its ability to produce a native image (a stand alone binary executable) for a Java application through Ahead of Time compilation. After normal compilation, GraalVM produces a single binary file that contains everything needed to run the Java application. This binary file starts a small as 10 MB – and it does not need anything else to run. No JRE or any form of Java Run Time. It just runs as native applications – because that is what it is. Startup time is every short and memory footprint is very small – ideal characteristics for Serverless Functions and dynamically scalable containerized applications. Because AOT is applied instead of JIT, there will be no run time optimizations to the application – which for serverless use cases is typically not a big loss at all. Quarkus and Micronaut are frameworks that make great use of GraalVM to produce even faster startup and smaller runtime footprint.

Oracle offers GraalVM in two flavors: a community edition which is based on open source and is offered for free and the enterprise edition which is paid for and offers 24/7 support and enhanced optimizations in the native image as well as improved security. For Oracle Cloud users, GraalVM Enterprise Edition is included in their subscription. Here is the global price list for GraalVM Enterprise Edition – a short inspection suggests a price of $12K to $19K per processor.

The big question around GraalVM is: will Oracle make the Community Edition sufficiently more appealing than OpenJDK with HotSpot to build up real traction and will it not bring too many goodies to the Enterprise Edition? To be fair: GraalVM offers many attractive features and it seems quite reasonable that Oracle stands to make some money for that effort and the value it delivers. Another question: will Oracle be able to follow the evolution of the Java language in GraalVM – such as the language enhancements discussed in the next section. It took until 19th November of 2019 before GraalVM provides full Java 11 support.

Note: GraalVM has been promoted from a research project to a ‘real product’ and the team around GraalVM is growing rapidly. This includes product management in addition to probably more developers and support engineers. GraalVM is serious business for Oracle.

Java Evolution

Java execution is still quite special: the runtime optimization performed by the JVM (C2 JIT compiler) provides many times better performance than static compilers.

Compatibility – old code must still run. Still the platform managed to absorb Generics, Lambdas and Streams, a Modular system. What are driving forces? Changing hardware, changing challenges and changing software [ how other languages do things].

• Project Amber – productivity oriented language features ( Local Variable Type Inference (JDK 10), Switch Expressions (JDK 12), Text Blocks (JDK 13), Concise class declarations (records), Sealed types, Pattern Matching)
• Java Fibers  and Continuations – Project Loom – concurrency, light weight focused at scalability: “A light weight or user mode thread, scheduled by the Java virtual machine, not the operating system. Fibers are intended to have very low footprint and have negligible task-switching overhead. You can have millions of them! Fibers allow developers to write simple synchronous/blocking code that is easy read, maintain, debug and profile, yet scales. Project mantra: Make concurrency simple again” Fibers are built on top of continuations – a low level construct in the HotSpot VM.
  (it is to be decided whether continuations themselves in their own right are to be exposed to developers). Associated terms: yield, async, (carrier) thread, executor, Promise, await, park and unpark. Watch: https://www.youtube.com/watch?v=lIq-x_iI-kc 
• Project Valhalla – “reboot the layout of data in memory” –  value types and specialized generics – benefitting from modern hardware – not everything needs to be an object – getting more instructions per CPU cycle by removing the memory [pipeline] bottleneck (first release deep into 2020)
• Project Panama – allow easy access to Java developers to native libraries – go beyond JNI (improve on the complexity, lack of JIT optimization, exchanging of native structs and off-heap data structures) and make a native library accessible in Java through JDK generated Interface. This hides away most of the native aspects of what is still a native library (see: https://www.youtube.com/watch?v=cfxBrYud9KM and read Project Panama home page https://openjdk.java.net/projects/panama/) Note: some overlap with GraalVM interoperability. Early access builds are available for Panama.

Books

Three special  book tips:

• Refactoring – edition 2 Martin Fowler, Kent Beck – december 2018 – https://martinfowler.com/articles/refactoring-2nd-ed.html
• Apprenticeship Patterns by Adewale Oshineye, Dave Hoover  – https://www.oreilly.com/library/view/apprenticeship-patterns/9780596806842/
• Java by Comparison Become a Java Craftsman in 70 Examples by Simon Harrer, Jörg Lenhard, Linus Dietz – https://pragprog.com/book/javacomp/java-by-comparison

And a website:

• https://cleancoders.com/

Tools

Below is a fairly random list of tools, sites, services and technologies that came to my attention during this Devoxx 2019 conference. They seem interesting, I would like to try them out. Most of them are as yet unknown. If you can recommend any – do let me know!

• DeckDeck Go – https://deckdeckgo.com/ Presentation Editor

• Karate (DSL) https://github.com/intuit/karate – open-source tool to combine API test-automation, mocks and performance-testing into a single, unified framework

• K3S Light weight Kubernetes https://k3s.io on Ubuntu; Similar: https://microk8s.io/

• Kind – Kubernetes in Docker- https://github.com/kubernetes-sigs/kind

• nip.io – Dead simple wildcard DNS for any IP Address; allows you to do that by mapping any IP Address to a hostname

• lvh.me

• Dive – inspecting and comparing Container Image (layers) – https://github.com/wagoodman/dive

• Newman – programmatic execution of Postman Test Collections

• WireMock is a simulator for HTTP-based APIs (http://wiremock.org/)

• http://rest-assured.io/ – testing REST APIs from java

• GitBook

• JDeferred – https://github.com/jdeferred/jdeferred – Java Deferred/Promise library similar to JQuery.

• Dive – inspecting container image layers – https://github.com/wagoodman/dive

• Kind – https://github.com/kubernetes-sigs/kind Kubernetes in Docker

• Testcontainers – Java gebaseerd programmatisch starten en manipuleren van containers op local or remote docker host

• QuickPerf – extension of JUnit – voor testen van performance gerelateerde applicatie-aspecten (in een heel vroeg stadium)

• Jbi – smart container image generation voor Java applicaties; ook : Skaffold (Container Pipeline), Helm (deployment van K8S applicaties) en Kustomization (omgevingspecifieke overrides van K8S yaml files)

• Google Cloud Run – very simple ‘run my container’ service

• Buildpacks – Buildpacks build apps in a consistent, repeatable way. Buildpacks know what a specific application type needs in terms of its runtime. The big idea behind buildpacks? Building containers from source code should be completely automated.

• Microbenchmarking the JVM:  https://openjdk.java.net/projects/code-tools/jmh/  (see https://www.youtube.com/watch?v=5AFgNuGwLos for a session that includes a demo)

• Jest – JavaScript testing – https://jestjs.io/ 

• Chai is a BDD / TDD assertion library for node and the browser that can be delightfully paired with any javascript testing framework. – https://www.chaijs.com/

• Apache Beam – data pipeline coordination

• ArrangoDB – multi model database

• Fairing

• Storybook – https://storybook.js.org/ – Storybook is an open source tool for developing UI components in isolation for React, Vue, and Angular. It makes building stunning UIs organized and efficient.

• Portecle – is a user friendly GUI application for creating, managing and examining keystores, keys, certificates, certificate requests, certificate revocation lists and more – http://portecle.sourceforge.net/ (that is HTTP not HTTPS)

• OpenSSL – OpenSSL is a robust, commercial-grade, and full-featured toolkit for the Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocols. It is also a general-purpose cryptography library. – https://www.openssl.org/

• NMap – https://nmap.org/ Nmap (“Network Mapper”) is a free and open source (license) utility for network discovery and security auditing. It was even featured in twelve movies, including The Matrix Reloaded, Die Hard 4, Girl With the Dragon Tattoo, and The Bourne Ultimatum.

• GopherJS – browser playground for Go(Lang) – and: transpile Go application to JavaScript – https://github.com/gopherjs/gopherjs 

• Anthos – Google’s K8S based Cloud Native platform (istio, ..)

• Kubernetes Security:

• Kube-bench – Checks whether Kubernetes is deployed according to security best practices as defined in the CIS Kubernetes Benchmark – https://github.com/aquasecurity/kube-bench

• Clair – Clair is an open source project for the static analysis of vulnerabilities in appc and docker containers. – https://coreos.com/clair

• Falco (CNCF, find rogue workloads, checks all sys calls – including SSH calls and disk writes), Falco is an open source project for intrusion and abnormality detection for Cloud Native platforms such as Kubernetes, Mesosphere, and Cloud Foundry. Detect abnormal application behavior. – https://falco.org/ 

• Sonobuoy a diagnostic tool that makes it easier to understand the state of a Kubernetes cluster by running a set of Kubernetes conformance tests and other plugins in an accessible and non-destructive manner.- https://github.com/vmware-tanzu/sonobuoy 

• Harbor (is an open source container image registry that secures images with role-based access control, scans images for vulnerabilities, and signs images as trusted.),

• kube-hunter an open-source tool that hunts for security issues in your Kubernetes clusters. It’s designed to increase awareness and visibility of the security controls in Kubernetes environments.- https://kube-hunter.aquasec.com/

• open policy agent – www.openpolicyagent.org

• https://github.com/codesenberg/bombardier Bombardier – http benchmarking

• https://nivo.rocks/ – visualization with React and D3.js

• Kiali – Service mesh observability and configuration – https://www.kiali.io/ 

These come on top of the more usual suspects such as Helm, Prometheus, Jaeger, Grafana, Maven, JUnit. Java Lambdas and Streams should be every day tools for Java Developers by now.