Abstract

Problem Statement: Despite the acknowledged importance of quantitative security assessment in secure software development, current literature still lacks an efficient model for measuring internal software security risk. Existing models are either not practical, since they are not fully-automated and operationalized in the form of a tool, or not reliable, since they are based on subjective parameters, while they also lack elaborate evaluation on empirical data.

Contributions: To this end, in the present paper we propose a hierarchical Security Assessment Model (SAM), able to assess the internal security level of software products based on low-level indicators, i.e., security-relevant static analysis alerts and software metrics. The model, following the guidelines of ISO/IEC 25010, and based on a set of thresholds and weights, systematically aggregates these low-level indicators in order to produce a high-level security score that reflects the internal security level of the analyzed software. The proposed model is highly practical, since it is fully-automated and operationalized in the form of a standalone tool and as part of a broader CASE platform. In order to enhance its reliability, the thresholds of the model were calibrated based on a repository of 100 popular software applications retrieved from Maven Repository. In addition, its weights were elicited in a way to chiefly reflect the knowledge expressed by the Common Weakness Enumeration (CWE), through a novel weights elicitation approach grounded on popular decision-making techniques.

Evaluation: The proposed model was evaluated through a set of experiments conducted on a large repository of 150 open-source software applications retrieved from GitHub and 1200 classes retrieved from the OWASP Benchmark. The results of the experiments revealed the capacity of the proposed model to reliably assess internal security both at product- and at class-level of granularity, with sufficient discretion power. They also provide preliminary evidence for the ability of the model to be used as the basis for vulnerability prediction.

Novelty: To the best of our knowledge, this is the first operationalized and fully-automated security assessment model that can be found in the related literature. In addition, it is the only model that sufficiently avoids subjective information, and that was built and evaluated on such a large volume of empirical data (i.e. a broad repository of 250 software applications comprising approximately 20 million lines of code).

Web Page Content

In the present web page, the supporting material of the research paper that describes the aforementioned Security Assessment Model (SAM) are made publicly available. In particular, the following materials are provided:

1. The proposed Security Assessment Model
2. An alternative Security Assessment Model that omits software metrics
3. The supporting tools that can be used for assessing software applications using the proposed model
4. The data and the R scripts that were used for the calculation of the model thresholds and weights.
5. The detailed evaluation results of the proposed model

Please navigate through the Menu that is provided on the top right corner of the web page.