Netflix migrated to AWS a few years ago, and converted our previously monolithic Java application into microservices. Today, the JVM Ecosystem team supports ~3,000 git repositories from hundreds of engineers which produce ~9,000 artifacts (JARs) per day! Also, engineers use a variety of JVM languages such as Java, Kotlin, Groovy, Scala and Clojure.

In such an environment where software is constantly changing and new versions of a library should be distributed across thousands of projects, how can you know that your projects are using the right version of a given dependency? What if an OSS library introduces a security vulnerability and you need to make sure that no one is using it in your company? What if an internal library introduces a bad change and you need everyone to upgrade/downgrade? Automating these for hundreds or thousands of engineers is crucial. 

At Netflix, engineers are not immune to the cost of dependency updates. Library owners publish new versions of their code without a comprehensive understanding of the organizational impact. Application owners ingest new library versions that can fail in obvious or subtle ways, leading to decreased confidence and slower organizational velocity. But these are problems we understand, and tooling can help. This is why we created Nebula.

After years of evolving our build, we’ve developed a few conceptual models of dependency management and we continue to improve the experience for Library owners and consumers at Netflix. You can learn more about the work we are doing in Dependency Management at Scale.

In addition, the JVM ecosystem team manages and provides guidance on how to make the most out of the build tool (Gradle) by providing Gradle Enterprise, Build Caching, Test Distribution and in-house solutions to speed up and have reproducible builds. We have partnered with companies such as Gradle and Varnish Software and are keeping close collaboration to accomplish some of this work.