The benefits of participating in the OpenJDK Quality Outreach Program
FOSS Projects and OpenJDK collaborating for a more robust Java platform.
The OpenJDK Quality Outreach Program
I’d like to tell you about an amazing OpenJDK Program that FOSS Java project maintainers should consider joining, to help guarantee that the Java Platform remains the best and most stable platform for software developers. An introduction to the OpenJDK Quality Outreach Progam (QOP), which is run by the OpenJDK Quality Group is available at the following link.
Joining the OpenJDK Quality Outreach Program is easy, and the benefits are real. The above link also has instructions for joining the program in a section titled “How to join the Quality Outreach Program”.
Once you join the program, and actively report your test results against early access versions of the OpenJDK, the benefits begin. Your test suite(s) become part of the overall suite of tests that can help validate the quality of the next OpenJDK release.
It’s that simple. You run the tests of your open source project against early access builds that can be easily setup if your project has automated builds setup with GitHub Actions.
Worst case, you never find a problem but still contribute to your own confidence level that the next release of the OpenJDK will work fine with your project. Win!
Best case, you discover a real problem that you report in time for the OpenJDK Core team to submit a bug fix before the next release. Win!
Eclipse Collections is an active member of QOP
Eclipse Collections has been actively reporting testing status against early access versions of the OpenJDK for several years. We email David Delabassee and he updates the project status on the OpenJDK Quality Outreach Program Wiki.
Nikhil Nanivadekar and I are the co-project leads for Eclipse Collections. We take turns reporting testing status to David and the OpenJDK Quality Outreach Program.
Early Access testing with the OpenJDK
I began testing GS Collections, before it became Eclipse Collections, with early access binary versions of the OpenJDK for the much anticipated Java 8 release. I was a member of the JSR 335 Expert Group, so was actively testing and reporting my experience to the Expert Group, testing Lambdas and Java Streams with GS Collections. Among other things, I discovered and reported a performance issue related to parallel Java Streams and non-JDK RandomAccess List implementations. I blogged about this experience at the following link.
The parallel Stream performance problem I discovered was verified and addressed with a fix in time for the JDK 9 release. The fix resulted in the creation of a class named RandomAccessSpliterator.
Enter the OpenJDK Quality Outreach Program
I scanned the quality-discuss mail list archive to see if I could find when the OpenJDK Quality Outreach Program became a formal program. The mail list and Quality Group dates all the way back to 2007. I discovered this email from Rory O’Donnell in August 2014 that looks like the beginning of the expansion of QA Outreach for the OpenJDK. This email corresponds with this Quarterly Report from Q1 2014.
Screenshot from above link
By 2015, the Quality Outreach Program was moved to the OpenJDK Quality Group as mentioned in this email from Rory. There were 48 participating FOSS projects at that time.
Based on a manual counting of projects on the Quality Outreach Program wiki today, the program now has 168 FOSS projects participating.
I’m going to share some discoveries in the sections that follow that the Eclipse Collections project has made during Early Access testing of OpenJDK releases over the past few years. I hope these stories encourage more FOSS Java projects to join the program.
A discovery in JDK 15
During testing of the Early Access version of JDK 15, Nikhil discovered an issue with a default isEmpty method that was added to CharSequence. The new default method clashed with a default isEmpty method we had defined much earlier on a type named PrimitiveIterable.
Nikhil reported the issue to the OpenJDK Quality Outreach Program. We decided the fix to the problem should happen in Eclipse Collections so we added implementations of isEmpty to all concrete types where the two default implementations resulted in ambiguity.
Stuart Marks wrote a great blog describing the problem we discovered with CharSequence.isEmpty.
The problem we discovered had more far reaching implications to the testing of default methods when they are added to existing interfaces in the OpenJDK or other widely used JVM Languages and Libraries. What we learned was that since the default method feature was introduced in Java 8, there exists the possibility for collisions of default and abstract methods to happen in diamond interface hierarchies with methods with identical signatures. Stuart describes this in depth in the article above.
Discoveries in JDK 21
During recent testing of the Early Access version of JDK 21, I discovered two issues. One issue was similar to the issue we discovered in JDK 15. New default methods were added along with new interfaces in JEP 431. The new default methods clashed with existing abstract methods we had defined in Eclipse Collections. The entire Java community was given a heads up on the potential impact of this JEP being included in JDK 21. This heads up helped me spot and understand the issue quickly. Thank you!
We had expected potential compilation issues in Eclipse Collections with the addition of Sequenced Collections in JDK 21. We have had getFirst and getLast methods defined on the OrderedIterable interface in Eclipse Collections for many years, and on RichIterable since version 1.0 of the library. The compilation issues we encountered may result in us releasing a new version of Eclipse Collections (12.0) that existing users of Eclipse Collections will have to upgrade to in order to use the library with JDK 21. I consider this an acceptable cost for the continued improvement of the Collections Framework in the JDK.
Update (July 8, 2023): It looks like a new release will not be necessary for Eclipse Collections to work with JDK 21. It looks like the issue with getFirst and getLast is a compilation only issue. I validated this looking at GitHub Actions builds using JDK 21 and Eclipse Collections 11.1 in the Eclipse Collections Kata and BNY Mellon CodeKatas Repos. This is good news! Thanks to Nicolai Parlog for asking me the question!
There was a second issue that I encountered that was unexpected, and has resulted in a fix being applied in the JDK itself. We have a battery of serialization tests in Eclipse Collections, that wind up testing some of the serialization of JDK types like ArrayList, HashSet, HashMap. We have never seen a failure for JDK types in our serialization tests. In our automated tests, I was seeing a failure of serialization for LinkedHashMap. I reported the failure I was seeing in our tests to the OpenJDK Quality Outreach Program. It turns out I had discovered and reported a real issue that has been since fixed for JDK 22 and backported for JDK 21.
Contributing to Java’s continued stability
I encourage other FOSS maintainers to join the OpenJDK Quality Outreach Program and participate in Early Access JDK testing. The more we collaboratively test the early access Java Platform releases, the more we collectively guarantee Java’s continued stability. Java is evolving rapidly, and this is a good thing. Evolving safely without breaking existing functionality is much better.
I have championed the participation of two other FOSS projects at my current employer in the OpenJDK Quality Outreach Program.
Spring Kata
A great set of Spring and Spring Boot code katas from Java Champion, Chandra Guntur.
Code Katas
A great set of Java code katas maintained by Emilie Robichaud and Aqsa Malik, who also blog here on Medium.
Thank you for reading this blog! I hope you will consider adding your FOSS projects to the OpenJDK Quality Outreach Program. The folks in the OpenJDK Quality Group are very supportive and appreciative of testing and contributions, as you can see in some of the links above. I enjoy working in this community, and I hope you will too. Together we can help guarantee that Java remains the best and most stable platform to work with for software developers.
I am the creator of and committer for the Eclipse Collections OSS project, which is managed at the Eclipse Foundation. Eclipse Collections is open for contributions.
