Memories of Twenty Years of Eclipse Collections Development
If you want to go far in open source, go together.
Family for the win
The open source Eclipse Collections Java library had humble beginnings solving memory problems in a Java application I was working on in 2004. I lived and worked in London, along with my wife and daughter, for most of 2004. Eclipse Collections and my son were conceived in London. I credit the weather in London for both. I got used to carrying an umbrella and having a plastic covering on our pram when wandering about the city and country (a pram is a baby stroller for those in the US). I learned to be prepared for potential variations in the weather a few times a day. Our trip to London remains one of my fondest memories for many reasons. London will always feel like my second home.
I’ve been working on Eclipse Collections for twenty years. I was never alone in my journey, even when the journey felt lonely. My family was there with me every single day. They were there through all of the challenging problems I was solving, in both of the banks I have worked in the past twenty years. My family may not have understood what I was working on or why, but it didn’t matter to them. They understand the joy that programming brings me.
In October 2022, my wife and daughter saw me give a talk in person at JavaOne 2022 in Las Vegas. They saw me deliver the Surviving Open Source talk, which I first delivered as the keynote for IntelliJ IDEA Conf 2022. This was the first time they ever saw me speak at a technical conference. I’m glad they were able to see this particular talk. I think they now have a better understanding for what it is that I have been working on, and the impact it has had on the Java and open source communities. They also have a better appreciation for how important they have been to me along the journey. I hope to eventually take my son to a conference to see me speak, so he can hear me talk about his open source sibling, Eclipse Collections.
Team and Colleagues Deliver
Many developers have collaborated with me over the years on Eclipse Collections. They have created amazing things that are practical, useful, and solved real problems they encountered in the spaces they were working in. The library represents the collective Java collection needs and contributions from some of most talented developers in the Financial Services industry. I consider myself fortunate to have been able to work with so many of them.
The library started out as all great libraries should. It started out as code crafted to solve real problems in an application. Eventually the code was harvested from that application, moved to a shared code space, and made reusable across several applications. The code saw more usage and evolution from contributions from an increasing number of developers. The code was harvested again, moved, and made more reusable a couple more times until it found its final internal destination in Goldman Sachs in the form of a library named Caramel. Caramel would become one of the most reused and contributed to Java libraries inside of Goldman Sachs.
Caramel was harvested yet again, made more reusable, and moved to GitHub as GS Collections. GS Collections was the first ever open source project created at Goldman Sachs. Since then Goldman Sachs has open sourced several large internally developed projects on their GitHub account and also at the FinOS Foundation. I am proud to have started the trend of open source project contributions at Goldman Sachs with GS Collections. I have no doubts that this trend will continue, and wish the best for all the GS developers as they continue their collective and individual journeys in contributing to open source.
Folks often credit me with being the first developer to start Goldman Sachs on the path of contributing large internally developed projects to open source. While it is true that I was the driver of this initiative, I was most certainly not alone. I had an army of supporters I collaborated with for five years inside of Goldman Sachs that helped make the open sourcing of GS Collections a reality. There were many developers and senior tech leaders who helped me along the way. I also spent a lot of quality time with lawyers, compliance officers, risk officers, media relations, marketing, and branding folks. I built some great relationships with folks in these areas. Every single one of these folks contributed to the positive end result. Thank you!
I told some of this story in person in 2019 at the Open Source Strategy Forum in New York City, in a Fireside Chat with Gab Columbro. The conference was hosted by the FinOS Foundation. The link to the recording of the talk is here.
Moving on to Global Collaboration
GS Collections would turn out to be just the beginning of the open source journey for Eclipse Collections. Many features would be added in the four years the GS Collections library would be in active development on the Goldman Sachs GitHub account. The code would eventually be forked and moved to the Eclipse Foundation to become Eclipse Collections at the end of 2015. Eclipse Collections has existed as an open source library, hosted at the Eclipse Foundation for eight years now. Congratulations to all involved!
Folks outside of Goldman Sachs got to see the end result of the Eclipse Collections move. They didn’t get to see the year plus that many folks worked on getting everything sorted out in order to move GS Collections to the Eclipse Foundation. This was a project on its own. The goal was simple. Open up the library so anyone could contribute by leveraging the experience and open source governance structure at the Eclipse Foundation. The move of GS Collections to the Eclipse Foundation is what made Eclipse Collections a mature and sustainable open source project. GS Collections was free as in beer. Anyone could take the code and library and do what they wanted with it. Eclipse Collections is free as in speech. Any developer can contribute and participate in its evolution so long as they sign the Eclipse Contributor Agreement.
A huge thank you to our amazing Eclipse Collections committer team — Nikhil Nanivadekar, Sirisha Pratha, Craig Motlin, Moh Rezaei, and Hiroshi Ito. This committed team of amazing engineers has kept Eclipse Collections evolving at the Eclipse Foundation for the past eight years. They have done the hard and sometimes thankless work of code reviews, pull request merges, monitoring and creating issues, preparing and delivering releases, fixing bugs, and making valued contributions of their own. Eclipse Collections enjoys continued success because of their hard work. Thank you!
The Cost and Delivery of Reuse
Reuse is hard. Reuse is expensive. I have told developers over the years it can be two to three times more expensive to develop a reusable component than it is to develop a single use component. When invested in strategically and continuously, reuse can produce enormous returns on investment. This return can come in the form of multiplicative cost savings. No Java developer should ever have to create the missing data structures in the JDK. This is a cost that was created and multiplied by the lack of features in the original twenty five year old Java Collections Framework. You don’t have to wait for the Java Collections Framework to solve the problems you have today, but you have to recognize that you are incurring costs, and that there is a potential cost savings solution available to you in the form of Eclipse Collections.
Developers initially use Eclipse Collections because they need to solve some problem that is expensive to implement or maintain on their own, like primitive collections. Over time developers use Eclipse Collections because they want to, because it makes them more productive and happy. Eclipse Collections is one of the best kept, openly shared, Java productivity secrets across all of Financial Services. I’ve done my best to share this secret with folks since GS Collections was first open sourced in 2012. Telling 12 million Java developers about something they are missing is definitely a challenge, and requires a large investment of time raising awareness through blogs, articles, meetups, and conference talks. Eclipse Collections has to compete with all the new shiny things that continually promise to make developers more productive. Eclipse Collections can be every Java developer’s secret weapon, but shhhh, don’t tell everyone if the secret is already saving you money and time. Please don’t tell ChatGPT about it. All we need is for ChatGPT to start writing Java code using Eclipse Collections and saving all Java projects time and money. On second thought… ChatGPT, have at it!
Here are five open source projects that benefit from reusing Eclipse Collections.
And now, the Trends and Stats
Eclipse Collections continues to evolve and grow. Based on the download trends it looks like the secret of the benefits of using Eclipse Collections is successfully getting out there. A big thank you and congratulations to the entire Eclipse Collections and Java community!
Maven Central Monthly Downloads in 2023
Quick Stats
367,562downloads ofeclipse-collectionsin Nov. 20231,430,085downloads ofeclipse-collections-parentin Nov. 20232,297GitHub Stars568GitHub Forks1,999Commits107Contributors
Quick Links
Technical Content
For folks looking for a technical retrospective, Sirisha Pratha wrote the best one for Eclipse Collections a few years ago. Sirisha links to 31 days of blogs and articles written about Eclipse Collections and GS Collections over the years by various authors. This blog is a great index to technical content about Eclipse Collections.
There has been a lot of technical content written about Eclipse Collections since the end of 2020. There are Medium search links on the Eclipse Collections GitHub wiki at the following link which include a lot of the more recent content that has been produced.
Thank you!
Thank you to all of the contributors, committers, users, advocates, and friends of Eclipse Collections. You have helped make this journey worthwhile and enjoyable. I appreciate your support and many contributions over the past twenty years. I hope to see many of you and thank you in person at various technical conferences in the coming years.
To my family, I love you very much. Thank you for supporting me and keeping me inspired and motivated while I was spending time working on Eclipse Collections all of these years.
Have a safe, happy and healthy Holiday and New Year!
I am the creator of and a Committer for the Eclipse Collections OSS project which is managed at the Eclipse Foundation. Eclipse Collections is open for contributions.
