JVM Ecosystem Team
Netflix's JVM Ecosystem team provides the user experience for dependency management, building, packaging, and publishing JVM-based libraries and applications through providing tools, automation, and guidance to stunning colleagues.
Netflix migrated to AWS a few years ago, and converted our previously monolithic Java application into microservices. Today, the JVM Ecosystem team supports ~3,000 git repositories from hundreds of engineers which produce ~9,000 artifacts (JARs) per day! Also, engineers use a variety of JVM languages such as Java, Kotlin, Groovy, Scala and Clojure.
In such an environment where software is constantly changing and new versions of a library should be distributed across thousands of projects, how can you know that your projects are using the right version of a given dependency? What if an OSS library introduces a security vulnerability and you need to make sure that no one is using it in your company? What if an internal library introduces a bad change and you need everyone to upgrade/downgrade? Automating these for hundreds or thousands of engineers is crucial.
At Netflix, engineers are not immune to the cost of dependency updates. Library owners publish new versions of their code without a comprehensive understanding of the organizational impact. Application owners ingest new library versions that can fail in obvious or subtle ways, leading to decreased confidence and slower organizational velocity. But these are problems we understand, and tooling can help. This is why we created Nebula.
After years of evolving our build, we’ve developed a few conceptual models of dependency management and we continue to improve the experience for Library owners and consumers at Netflix. You can learn more about the work we are doing in Dependency Management at Scale.
In addition, the JVM ecosystem team manages and provides guidance on how to make the most out of the build tool (Gradle) by providing Gradle Enterprise, Build Caching, Test Distribution and in-house solutions to speed up and have reproducible builds. We have partnered with companies such as Gradle and Varnish Software and are keeping close collaboration to accomplish some of this work.
On the horizon
The JVM ecosystem team is ramping up on managing the JDK usage at Netflix, making sure that engineers can upgrade to new LTS versions with little or no friction, and making sure that produced libraries are compatible with existing applications using older or modern JDKs.
Blogs and Presentations
Dependency Management at Scale (Spanish)
Roberto Perez Alcolea talks about how Netflix manages complex dependency graphs within and between large-scale microservices.
A More Integrated Build and CI to Accelerate Builds at Netflix
Aubrey Chipman & Robert Perez Alcolea talk about Netflix's build ecosystem and how integration drivs acceleration and improves developer velocity.