The figure shows how data are used by our tools and how they eventually produce research results. The data in the figure are represented as plain text, while our tools are represented as rectangle text. The names of the tools are the same as the ones in our repository. The detailed build scripts and documentation are also in the repository.

In general, we first collect various types of data from the Rust ecosystem, and extract or generate ecosystem raw data. The data are all stored in the PostgreSQL database for fast query, which only takes at most 2 min to process over 140M transitive dependencies. By maintaining the database, we can do a lot more than RUF study in the Rust ecosystem (see extensibility in the home page for more). We further write SQL scripts to process the raw data to get research results that form the final version of our research paper. 

To mitigate RUF impacts, we also developed RUF Detector named cargo_ruf to detect given Rust projects to provide useful information on enabled RUF. Most importantly, it automatically tries to recover the package if it suffers from compilation failure introduced by enabled RUF. For compatibility concerns, it can also be integrated into Cargo, which is the Rust official package manager.

Reproduce Our Results

The figure shown above shows the input and output of each tool we developed in our research. The detailed configurations and build guidance can be seen in the GitHub repository. Also, to better reproduce the results, we provide docker image with the execution environment and database to reproduce the data. To make things work better, we also provide ecosystem raw data, so you don't have to download 100GB source codes, maintain the complete dependency index, etc.

Get started by downloading our source code here in the "Resouce Download" section.

EDG Accuracy

We validate our Ecosystem Dependency Graph (EDG) accuracy by comparing it with the official dependency resolution tool Cargo Tree from Cargo-1.63.0.

Benchmark Setup:

We resolute four types of dependency: build, common, optional, and target. Only development dependencies are omitted as they will not affect the runtime of programs, which is the same as our resolution rules. 

We first download source code from the official database Crates.io, and then use Cargo Tree to resolve dependencies in the real environment. After that, we will compare dependency items from Cargo Tree and our ecosystem dependency graph.

We treat each dependent version as a dependency and the sum of dependent versions as a dependency tree in the evaluation process. This is because we only care about whether a specific package version impacts the root package in the dependency tree rather than how it impacts the dependency tree.

We use four indexes to represent accuracy shown below. Tree Accuracy stands for the resolution accuracy of the entire dependency tree. Recall and Precision represent Right percentage in the standard dependency and resolved dependencies data set, respectively. F_{1}-score is the harmonic mean of recall and precision, which can represent the accuracy of resolution.