Towards quality-centric design and evaluation of big data cyber security analytics systems

Abstract

Big Data Cyber Security Analytics (BDCA) systems are a new breed of software systems that leverage big data technologies to collect, store, and analyse a large volume of security events data for detecting cyber-attacks. To detect sophisticated and complex cyber-attacks, many organizations are rapidly adopting BDCA systems to analyse a large volume of security events data in diverse formats from multiple sources such as network devices, software applications and honeypots. BDCA systems are a complex class of software systems that are expected to fulfil a certain class of quality attributes such as response time, accuracy, and scalability. Given the increasing volume, velocity, and heterogeneity of security events data, BDCA systems present unique design challenges and new research and development opportunities for providing suitable design and evaluation support. However, most of the research efforts have focused on algorithmic solutions (e.g., data filtering and feature selection) for optimizing response time, accuracy, and scalability of BDCA systems. Hence, there is an important need of research efforts aimed at providing suitable design knowledge (e.g., architectural tactics and design guidelines) for BDCA systems. More research efforts also need to be invested in exploiting the optimization opportunities (e.g., selection of components) offered by the architectural design of BDCA systems for optimizing response time, accuracy, and scalability. This thesis aims at contributing to the growing body of design and evaluation knowledge for BDCA systems by gathering/devising, implementing, and evaluating a set of quality-centric design approaches for optimizing response time, accuracy, and scalability of BDCA systems. This thesis advances the domain of BDCA systems’ design and evaluation knowledge by making the following contributions. • Design, conduct, and report a systematic literature review of the state-of-the-art BDCA systems to identify the most important quality attributes and codify architectural tactics for BDCA systems • Quantify the impact of architectural tactics on the accuracy and response time of a BDCA system through a systematically designed experimentation, which leads to the formulation of tactics-specific guidelines for designing BDCA systems • Present and evaluate a design approach for determining an architecture for a BDCA system at design time that offers optimal accuracy and response time • Present and evaluate an architecture-driven adaptation approaches for (re)composing a BDCA system at runtime with a set of components to ensure optimal accuracy and response time in the face of changes in the operating environment of the system • Present and evaluate a scalable fuzzy rule based approach to correlate security event data with the components of a BDCA system for (re)composing a BDCA system at runtime to ensure optimal accuracy and response time in the face of changes in the operating environment of the system. • Investigate the scalability of a BDCA system with the default and modified configuration setting of the underlying big data framework (i.e., Apache Spark) to explore and subsequently exploit configuration setting for optimizing scalability