In this page, we aim to detail the distribution of root cause on 3-level architecture of JavaScript-based DL system, i.e., Framework (i.e., TensorFlow.js), 3rd-party Library and Web Application. Meanwhile, we aim to summarize and highlight the main root causes introduced at each level, which can help improve the quality of JavaScript-based DL systems. The distribution of root cause at different levels is different, as detailed below.
(a) Distribution on TensorFlow.js Framework
The left figure shows the distribution of root causes on the framework level, i.e., TensorFlow.js. We can see that TensorFlow.js introduces the most faults (i.e., 389 faults), which are caused by various root causes. Specifically, the root cause Incorrect Code Logic is the most common and the symptoms of the faults it causes have various symptoms and are difficult to localize. Moreover, all Execution Environment Error (i.e., Device Incompatibility, Browser Incompatibility) are distributed in TensorFlow.js, and these root causes are specific to TensorFlow.js. Note that 85.25% of Dependency Errors is distributed in TensorFlow.js and many operators are not yet supported by TensorFlow.js. These results indicate framework developers should focus on detecting faults caused by Incorrect Code Logic before releasing a new version of the framework to ensure that the implementation logic of each module is correct. Meantime, quite a few dependency errors indicate that framework developers need to check and update the libraries that TensorFlow.js depends on in time. The unimplemented operator may result in many DL tasks cannot be executed, so framework developers should implement as many operators as possible to align with the mature PC-side framework (i.e., TensorFlow).
(b) Distribution on 3rd-party DL Library
In terms of 3rd-party DL library, they introduced 128 faults. Specifically, the Incorrect Code Logic and Tf.js Incompatibility (i.e., Incompatibility between 3rd-party DL Library and TensorFlow.js) are the main root causes, accounting for 67.19% (86/128). Additionally, there are quite a few faults related to models provided by 3rd-party library (i.e., Improper Model Attribute). These results show that in addition to focusing on the faults caused by the framework, the faults caused by the 3rd-party libraries also need to be paid attention to by developers or researchers. In particular, 3rd-party DL library developers should improve specific algorithms to avoid incorrect code logic errors, and need to improve the quality of pre-trained models to improve user experience. In addition, incompatibility with TensorFlow.js is another serious problem with 3rd-party DL libraries. As a result, only a specific version of TensorFlow.js can be used, which brings great inconvenience to users. Such issues should arouse the attention of developers.
(c) Distribution on Web Applications
In terms of web applications, they introduced 183 faults. Specifically, most faults are caused by API Misuse, including missing or redundant calls to an API, wrong API names, invalid API input/parameters (i.e., type/shape/value error). Invalid input/parameters in particular account for the largest proportion. This indicates that application developers are confused about API usage, especially some APIs that are only supported in certain environments. Besides, Data/Model Inaccessibility is the second common, which is specific to JavaScript-based DL systems. This is usually because developers are unfamiliar with the features of browsers and UI frameworks. Similar to framework and 3rd-party library, Incorrect Code Logic is also a common root cause. Apart from that, Import Error also causes quite an error. These results indicate application developers should use the API carefully, such as understanding the usage of the API based on the official documentation before using the API. Meantime, since data and models are the foundation of DL applications, developers should first ensure their accessibility by checking model URL/path to avoid data/model inaccessibility. Overall, the faults caused by Incorrect Code Logic comes from all 3 levels and accounts for the most, which should be the focus of all developers’ attention.