Industrial Software Engineering: Agile Model/Tools for Mobile & Big Data app

For Big Data and Open Source Software (OSS)

• K. S. Yim, "Evaluation Metrics of Service-Level Reliability Monitoring Rules of a Big Data Service," In Proceedings of the IEEE International Symposium on Software Reliability Engineering (ISSRE), pp. 376-387, October 2016. (Acceptance Ratio: 35% = 45/130) [Paper] [Slide] [BibTeX]
  • (Abstract) This paper presents new metrics to evaluate the reliability monitoring rules of a large-scale big data service. Our target service uses manually-tuned, service-level reliability monitoring rules. Using the measurement data, we identify two key technical challenges in operating our target monitoring system. In order to improve the operational efficiency, we characterize how those rules were manually tuned by the domain experts. The characterization results provide useful information to operators supposed to regularly tune such rules. Using the actual production failure data, we evaluate the same monitoring rules by using standard metrics and the presented metrics. Our evaluation results show the strengths and weaknesses of each metric and show that the presented metrics can further help operators recognize when and which rules need to be re-tuned.
• K. S. Yim, “Norming to Performing: Failure Analysis and Deployment Automation of Big Data Software Developed by Highly Iterative Models,” In Proceedings of the IEEE International Symposium on Software Reliability Engineering (ISSRE), pp. 144-155, November 2014. (Acceptance Ratio: 25% = 31/124) [Paper] [Slide] [BibTeX]
  • (Abstract) We observe many interesting failure characteristics from Big Data software developed and released using some kinds of highly iterative development models (e.g., agile). ~16% of failures occur due to faults in software deployments (e.g., packaging and pushing to production). Our analysis shows that many such production outages are at least partially due to some human errors rooted in the high frequency and complexity of software deployments. ~51% of the observed human errors (e.g., transcription, education, and communication error types) are avoidable through automation. We thus develop a fault-tolerant automation framework to make it efficient to automate end-to-end software deployment procedures. We apply the framework to two Big Data products. Our case studies show the complexity of the deployment procedures of multi-homed Big Data applications and help us to study the effectiveness of the validation and verification techniques for user-provided automation programs. We analyze the production failures of the two products again after the automation. Our experimental data shows how the automation and the associated procedure improvements reduce the deployment faults and overall failure rate, and improve the feature launch velocity. Automation facilitates more formal, procedure-driven software engineering practices which not only reduce the manual work and human-oriented, avoidable production outages but also help engineers to better understand overall software engineering procedures, making them more auditable, predictable, reliable, and efficient. We discuss two novel metrics to evaluate progress in mitigating human errors and the conditions indicating points to start such transition from owner-driven deployment practice.
• K. S. Yim, D. Hreczany, and R. K. Iyer, "HTAF: Hybrid Testing Automation Framework to Leverage Local and Global Computing Resources," Lecture Notes in Computer Science (LNCS), 6784:479-494, June 2011. (Impact Factor: 0.97)
  • (Abstract) In web application development, testing forms an increasingly large portion of software engineering costs due to the growing complexity and short time-to-market of these applications. This paper presents a hybrid testing automation framework (HTAF) that can automate routine works in testing and releasing web software. Using this framework, an individual software engineer can easily describe his routine software engineering tasks and schedule these described tasks by using both his local machine and global cloud computers in an efficient way. This framework is applied to commercial web software development processes. Our industry practice shows four example cases where the hybrid and decentralized architecture of HTAF is helpful at effectively managing both hardware resources and manpower required for testing and releasing web applications.

Software Engineering Infrastructure

• K.S. Yim, J. Shin, "Fuzz testing of smartphones and IoT devices," Technical Disclosure Commons, (April 08, 2019) https://www.tdcommons.org/dpubs_series/2123
• K. S. Yim, Y. Ma, "Real-time scheduling for software testing", Technical Disclosure Commons, (April 11, 2018) https://www.tdcommons.org/dpubs_series/1162
• K. S. Yim, "Distributed and Collaborative Test Scheduling to Determine a Green Build", Technical Disclosure Commons, (September 20, 2018) https://www.tdcommons.org/dpubs_series/1519