ICON: Inferring Temporal Constraints from Natural Language API Descriptions

Temporal constraints of an Application Programming Interface (API) are the allowed sequences of method invocations in the API. These constraints govern the secure and robust operation of client software using the API. However, in practice, most APIs do not come with formal temporal constraints. In contrast, these constraints are typically described informally in natural language API documents, and therefore are not amenable to existing tools for checking formal temporal constraints. The goal of this work is to assist developers to construct API clients that comply with temporal constraints of the API through the inference and formalization of these constraints found in natural language API documents. Since API documents are often verbose, manually identifying and writing formal temporal constraints can be prohibitively time-consuming and error-prone. To address this issue, we propose ICON: an approach based on Machine Learning (ML) and Natural Language Processing (NLP) for identifying and inferring formal temporal constraints. To evaluate our approach, we apply ICON to infer and formalize temporal constraints from the Amazon S3 REST API, the PayPal Payment REST API, and the java.io package in the JDK API. Our results indicate that ICON is effective in identifying temporal constraint sentences (from over 4000 human-annotated API sentences) with the average precision, recall, and F-score of 79.0%, 60.0%, and 65.0%, respectively. Furthermore, our evaluation also demonstrates that ICON achieves an accuracy of 70% in inferring and formalizing 77 temporal constraints from these temporal constraint sentences.


ARTIFACTS
Subpages (1): Subjects