Divide and Conquer Feature-Rich APIs, Categorically!
A simple teaching strategy to help developers learn large APIs.
I wrote a programming book titled Eclipse Collections Categorically: Level up your programming game. There are important lessons in this book for developers who program in any programming language. I hope one or more of these lessons will be a compelling reason for you take a look at the book. The front matter of the book is available in the online reading sample for free if you want to get a taste of what’s inside!
What’s in this book for you?
Buckle up. This is one book you might not want to overlook.
The Eclipse Collections library provides open source collections for Java that follow the “Tell Don’t Ask” principle. You tell the collections directly what it is that you want to be done and they do those things for you. Eclipse Collections has hundreds of efficient solutions to common problems Java developers face every single day. This is in contrast to the standard Java collections which require asking for a Java Stream to do interesting and useful things with the data in a collection.
The Eclipse Collections Categorically book was organized to help developers divide and conquer the hundreds of methods in the library to efficiently match problems to solutions. The book is more than a book about data structures and algorithms. It is a book about categories of algorithms, that are available across data structures. The book defines recognizable categories of problems that Eclipse Collections types can solve. The book includes 200+ examples using specific methods available on Eclipse Collections types in each of the categories, in full color with syntax highlighting.
Here are the categories of things explained in the book that you might need to do with collections:
🧮 Count things (Chapter 3)
🧪 Test things (Chapter 4)
🔍 Find things (Chapter 5)
🚰 Filter things (Chapter 6)
🦋 Transform things (Chapter 7)
🏘️ Group things (Chapter 8)
📊 Aggregate things (Chapter 9)
🔌 Convert collections to different collection types (Chapter 4)
🔄 Iterate over things and do stuff (See Appendix A)
Eclipse Collections Categorically will help you find methods and data structures that can help you solve problems in each of these categories.
The nine categories (counting, testing, finding, filtering, transforming, grouping, aggregating, converting, and iterating), cover all 134 of the non-overloaded methods available in the RichIterable parent type in Eclipse Collections. Click the RichIterable link to see how the methods have been organized on the type in the source code. This categorization works in Javadoc and in IntelliJ IDEA structure view to help you find things. I’ve blogged about this previously.
There is another category in the book that is specific to Set types, covered in Chapter 10.
🔢 Set operations (Chapter 10)
The Set operations covered in the book includes union, intersect, difference, symmetricDifference, and cartesianProduct. The Eclipse Collections library includes additional Set operations. These five methods were good enough to cover in the book, as they cover the most common use cases.
Why should you care about Eclipse Collections?
There are several reasons to care about Eclipse Collections, and all have to do with reducing costs. The first, and probably the most overlooked cost is the “Cost to read code.” The book explains in detail, with 200+ syntax highlighted color code examples this particular cost.
Cost to read code
If you believe that reducing the cost of reading code for developers is important, then you should care about Eclipse Collections. Most of the code examples in the book assess a cost of reading the code with a number of books 📕.
There are examples that solve problems using Eclipse Collections, Java Stream and Collectors in the book. I wrote the examples in the book using Java 21. The book was published in paperback a couple of days before Java 24 was GA, so there are no Gatherer examples to compare with in the book. I may write some Gatherer examples in a future edition of the book. I wrote a blog recently that demonstrated using a couple of methods on Gatherers and their alternative in Eclipse Collections. I did not assess a cost to read the code in these examples, but you can decide which examples have a higher cost to read.
Memory cost
I explain in the preface of the book, why I created Eclipse Collections. One of the reasons included having to solve problems I faced over twenty years ago working in a memory constrained space. Whether you work with large or small Java heaps, if the cost of memory is a concern for you, then you should be aware of the memory saving benefits of Eclipse Collections. Even if you already use Eclipse Collections, Eclipse Collections Categorically can help you discover, learn and leverage more of the amazing methods you may not know exist on the Eclipse Collections types.
Project Valhalla will hopefully one day arrive and help us all. I couldn’t wait for Valhalla to solve the problems I faced in financial services applications, and you don’t have to either. Eclipse Collections has an extensive set of primitive collections and is an alternative available today that works in every version of Java since Java 8. If you are learning for the first time that Eclipse Collections can help you save memory at runtime, then read the following free blog before buying the book.
Performance cost
Performance is where you need to do your own homework. If you are concerned that the Java Collection or Stream code you are writing might be a performance bottleneck, then you should write some application level benchmarks with alternatives like Eclipse Collections and find out for yourself if you can improve any unacceptable performance.
I’ve written a lot of micro-benchmarks with Eclipse Collections that will probably be of no specific help to you. The micro-benchmarks helped me and other developers tune the Eclipse Collections library in specific places. This tuning may or may not help applications you work on noticeably in specific situations. I prefer to see application level benchmarks that show that something we did in Eclipse Collections over the past two decades has a benefit. Here’s one blog that compares Data Frame libraries used to execute the One Billion Row Challenge (#1BRC) written by . It demonstrates some specific benefits Vlad was able to measure in his open source Java Data Frame library called dataframe-ec. YMMV.
Costs and benefits of parallelizing code
There are examples in the book that demonstrate ways to parallelize code. I do not provide any benchmarks in the book to show that performance of any of the use cases improved. I provided some tips on how to determine whether something is worth parallelizing, along with some warnings about taking an undisciplined approach to parallelizing code.
The book has some parallel examples that show you how to do things, but it can’t tell you if you should. Only you can determine that, for your use cases, verified with your own benchmarks.
Here is a free blog with some random micro-benchmarks that hopefully do nothing more than make you curious to try things on your own. Writing parallel code and hoping it will magically go faster can be expensive. As I say in the book, “the dragons are real.”
Energy Cost
For years, I wondered what effect the memory and performance savings we factored into Eclipse Collections might have on energy cost. I never measured the cost of energy in micro-benchmarks myself, but I discovered the past few years that there were others that had. The following is the most recent blog I’ve seen that measures energy cost of different collections for specific operations. These benchmarks may be of no real use to you, but might be helpful to think about if “going green” is of interest to you. Again, measure the benefits for your own application code. There is a framework the author mentions in this blog, called JoularJX, that may be worth checking out. Note: I have not yet tried this particular framework, so can’t endorse it directly.
Not a Java programmer?
There are a lot of programming languages available to program in besides Java. You might be happy as a clam programming in Rust, Go, Python, Kotlin, Scala, JavaScript, TypeScript, Scala, Groovy, Swift, C#, C++, etc. Maybe you don’t need or care about a third-party collections library for Java. What could be in this book that could help you improve the programming language(s) you work in, or the way you write your own code?
Spoiler alert: message categories.
I didn’t include Smalltalk in this list of programming languages. There is a reason. Smalltalk has a feature built in the language and tooling called message categories, which I refer to as method categories in the book. I don’t believe any of the file based languages listed above have support for this feature (please, correct me if I’m mistaken). Smalltalk has had message categories since before 1980. Please see page 42 of the book Smalltalk-80: The Language, by Adele Goldberg and David Robson, for the answer why.
Even though Smalltalk has message categories, current and former Smalltalkers can still learn some things from this book. I say this as a former Smalltalk developer who learned something very important about message categories that I didn’t appreciate 30 years ago when I learned Smalltalk. We current and former Smalltalkers are lucky to have message categories built into Smalltalk. We can and should focus on using them more effectively, for the humans that have to read and learn our code. I have an appendix in the book dedicated to Smalltalk, with a visual hint showing how message categories in the Collection base type in Pharo Smalltalk might be improved by better balancing the the counts in some categories.
If you don’t program in Smalltalk, and love your programming language, then you might be able to help yourself and your favorite programming language to understand what might be missing that can help organize feature-rich APIs. This can help make your favorite programming language easier for developers to learn in the future. We can make all programming languages better, if we learn from each other.
The book can offer you and the designers of your favorite programming languages an advanced class on information chunking and how it can aid human comprehension.
But wait, there’s more…
Imagine what it would be like if a former Smalltalker spent twenty years of his life working on a comprehensive collections library in Java. Mirror mirror on the wall, who’s the… what the… oh no... darn…
I was not alone in my journey to make Java more productive. I doubt anyone would have enough time to learn everything that has been done in Eclipse Collections. I’ve worked on the library for twenty-one years, and not even I know everything that is in there. There is over one million lines of code in the library. No one can or would want to read all that code line by line and remember everything.
Eclipse Collections Categorically shows you how to take a million plus lines of code and break it down into symmetric patterns that can help you navigate and discover things.
Eclipse Collections does not have everything, but it has enough things to make it interesting for so many use cases. Eclipse Collections may help you become a more constructive and instructive programmer in other programming languages. We can do so much better to improve our programming languages, environments, libraries, and tools. We settle early very often due to costs. We should learn to expect more, and not just settle for what we’re given. We’ve occasionally got to be willing to roll up our sleeves and get involved. Java didn’t have lambdas. Collaborative folks got to work, and now we have lambdas in Java, since 2014. Java needed better collections. Collaborative folks worked for twenty-one years and are still working on stuff, and we got Eclipse Collections. You can get involved if you want as well. There are plenty of problems to solve.
That’s how stuff works. When people need things, they roll up their sleeves, and they get to work. It’s not rocket science. It’s fun to break through walls no one else has gone through once in a while. First, you need the determination to see something beyond that wall. I saw lambdas in Java clearly in 2004. People I worked with probably thought I was crazy. All they saw were these ugly anonymous inner classes I was willing to write code with. Build an ark, get a lambda flood. :)
Why did I write a book about Eclipse Collections?
I wanted to find a more efficient way to teach developers how to leverage a million plus lines of productivity enhancing, memory efficient, Java collections code. I took a year plus off of work to write a book that does this. Writing the book led to an epiphany of the importance of method categories. It led to me discovering other things as well.
There are more things in the book that may surprise you. The inside of the paperback and hardback versions of the book are in full color. You may learn things completely unrelated to Java, collections, or anything to do with software development. In the following code example, you can learn some stats about generations and United States presidents.
And with that… my ten minutes are up. Thank you for reading! I hope you will check out the book, and get value from it for as long as you write code. Thank you in advance if you buy the book! 🙏
Note: The paperback and hardback versions of Eclipse Collections Categorically are now both available on Amazon. The eBook should be available on Amazon by June 30th. Other platforms and distribution channels will eventually follow.
I am the creator of and committer for the Eclipse Collections OSS project, which is managed at the Eclipse Foundation. Eclipse Collections is open for contributions. I am also the author of the book, Eclipse Collections Categorically: Level up your programming game.
