Machine Learning Based Relevance Filtering of Shared Cyber Threat Intelligence​

Due to the explosive growth of Internet cyber threats are evolving at an unprecedented pace, and Cyber Threat Intelligence (CTI) has become an important asset for organisations to the creation of situation awareness and safeguarding of their systems against new and emerging cyber threats. Moreover, in today's interconnected digital landscape organisations are employing dedicated platforms to facilitate the automated or semi-automated sharing of CTI. However, this CTI sharing introduces several challenges, and the most crucial challenge is to handle the plethora of shared threat information that may not be relevant to each organisation. Therefore, content based relevance filtering of CTI is necessary to facilitate the automated sharing of CTI. As Cyber Security and CTI involve sheer volume of information, Big Data and Machine Learning (ML) could prove a natural choice to bolster the effectiveness of CTI sharing. It is found in the literature that Machine Learning has been used in Cyber Security in various perspective, for example, A) Natural Language Processing (NLP) has been used to extract structured CTI from unstructured threat reports, or from unstructured CTI reports. Machine Learning and Deep Learning has been utilized to easily identify, collect, analyse, extract, integrate, and share cyber-threat intelligence from a wide variety of online sources. B) Various ML models have been used to detect anomalous behaviour and predicting cyber threats. However, as far our knowledge go, Machine Learning has not been applied in the relevance filtering of CTI. Various existing relevance filtering mechanisms are found in the literature, e.g. CTI is filtered based on specific key terms or attributes, domain tagging is applied for CTI filtering, where CTIs are classified into different domains related to different organisations, contextual information of business processes has been used to describe conditions under which a given CTI is actionable from an organization perspective. All these approaches suffer from their own limitations, e.g. keyword based relevance filtering error-prone, and demands expertise in order to define the proper keywords. Domain tagging and contextualization may only result into high level filtering. Moreover, all these approaches are labour intensive and require manual intervention that could be an obstacle for automated sharing of CTI. 

​This PhD research would investigate to develop a framework by applying Machine Learning techniques for relevance filtering in order to share CTI among organisations. The framework aims to be fully automatic that will analyse unstructured CTI reports and transform the CTI into structured format and then classify the structured CTI relevant to the organisations participating in CTI sharing. 

his PhD research would investigate to develop a framework by applying Machine Learning techniques for relevance filtering in order to share CTI among organisations. The framework aims to be fully automatic that will analyse unstructured CTI reports and transform the CTI into structured format (e.g. STIX/TAXII) and then classify the structured CTI relevant to the organisations participating in CTI sharing. Anticipated findings of this study are listed below: 

• ​A structured language to specify CTI requirements of the organisations in different domains. 

• ​ML models to classify CTI relevance tailored to the requirements of different organisations.  

• ​A conceptual framework for sharing CTI among participating organisations and implementation of a proof-of-concept prototype of the framework. 

• ​Comparative evaluation of the implemented prototype against existing works.​