PRQ2: Dependency Tree Changes

To investigate the frequency of the imperceptible dependency changes, we extend our dependency tree resolution to the time dimension, and capture the next DTC of a dependency tree by identifying the next released version of all packages in this tree that triggers the change of nodes or relations in dependency trees. A DTC can be identified if the tree changes, and the dependency tree after DTC can be recalculated on that release time.

We first collect all versions (1,010,075 versions belonging to 154,111 libraries) released in the most recent three months and calculate their DTCs from release time. The reason for selecting library versions within three months is because a longer time always leads to exponentially increased DTCs, especially for library versions with large dependency trees, according to our observation.

Besides, we excluded versions that (1) contain dependencies with missing released time or having abnormal dependency trees that cannot be resolved within a limited duration; (2) have no dependency or their dependency trees never change during the three months.

We finally filter out 581,192 library versions and over 34 million dependency trees of them from release to current time, and averagely, each library version has experienced 57 DTCs.

Based on DTCs, we further measure time intervals between them to represent DTC frequency. As shown in Fig below, generally each dependency tree only persists (i.e., unchanged) for less than 18 hours on average. Particularly, half of DTCs intervals are less than an hour, and 88% of DTCs intervals are less than a day. These findings unveil that dependency trees of NPM packages are under extremely frequent changes.

Therefore, we further look into the causes of such frequent DTCs. We analyze the release interval for each library and find that 4.80% and 48.85% of libraries have ever released multiple versions within a minute and an hour, respectively. These indicate that there are indeed some notable releases for some libraries, while most libraries are following proper version release frequency.

These findings proves that, apart from a small group of libraries that have frequent version release, the dependency tree itself exponentially amplifies the frequency of version releases, leading to frequent DTCs.

From the 34 million dependency trees we calculated, we find 33,806,301 DTCs in total, and we also extract the change points for each of them to analyze their location in dependency trees to unveil the change impact.

We find that 68.58% of affection points are from transitive dependencies and the rest 31.42% of them are from direct dependencies. However, according to the dependency trees that we have resolved in PRQ0, each dependency tree has 5.7 direct and 109 transitive dependencies on average. It seems that direct dependencies (4.97% of all dependencies) have a much higher portion in change points (31.42%). These indicate that version releases in direct dependencies have a higher possibility to trigger DTC. This is probably because when constructing the physical tree during installation, the higher dependencies (closer to root package) have a larger scope than lower ones, since higher dependencies are resolved ahead of lower ones and can affect the version selection of lower dependencies.

Besides, we also find that only 29.78% of change points are at leaf nodes in dependency trees, the rest 38.80% of change points are at transitive positions that are neither directly connect to root nor at leaf node in dependency trees, which means most of the change points are at higher positions in dependency trees and could lead to the replacement of the entire following sub-branch instead of a single leaf node in corresponding DTCs.

To investigate the initiative change on direct dependencies from developers, we filter out libraries that have changed their direct dependencies during version upgrades. We find that 70.82% libraries (812,662) have more than one version, among them, only 33.61% %of them (385,681) have ever introduced direct dependency changes. In total, 9,804,419 times of version updates (releasing new version) have been identified, and 29.17% of them (2,859,805) come with direct dependency changes.

On average, every 3.42 version releases bring one direct dependency update. 83.71% version releases (8,207,188 out of 9,804,419) are patch version releases (including pre-release tag changes), and the rest 16.29% are major or minor version releases {(2.96% and 13.33%, 290,289 and 1,306,942 out of 9,804,419. respectively). However, 44.68% major version upgrades have modified direct dependencies, while only 33.80% and 27.88% of minor and patch version releases have introduced direct dependency changes.