Please use this identifier to cite or link to this item:
https://hdl.handle.net/2440/129195
Citations | ||
Scopus | Web of Science® | Altmetric |
---|---|---|
?
|
?
|
Type: | Conference paper |
Title: | PUMiner: Mining security posts from developer question and answer websites with PU learning |
Author: | Le, T.H.M. Hin, D. Croft, R. Babar, M.A. |
Citation: | IEEE International Working Conference on Mining Software Repositories, 2020, pp.350-361 |
Publisher: | ACM |
Publisher Place: | New York, NY |
Issue Date: | 2020 |
Series/Report no.: | IEEE International Working Conference on Mining Software Repositories |
ISBN: | 9781450375177 |
ISSN: | 2160-1852 |
Conference Name: | 17th International Conference on Mining Software Repositories (MSR) (29 Jun 2020 - 30 Jun 2020 : Virtual) |
Statement of Responsibility: | Triet Huynh Minh Le, David Hin, Roland Croft, and M. Ali Babar |
Abstract: | Security is an increasing concern in software development. Developer Question and Answer (Q&A) websites provide a large amount of security discussion. Existing studies have used human-defined rules to mine security discussions, but these works still miss many posts, which may lead to an incomplete analysis of the security practices reported on Q&A websites. Traditional supervised Machine Learning methods can automate the mining process; however, the required negative (non-security) class is too expensive to obtain. We propose a novel learning framework, PUMiner, to automatically mine security posts from Q&A websites. PUMiner builds a context-aware embedding model to extract features of the posts, and then develops a two-stage PU model to identify security content using the labelled Positive and Un-labelled posts. We evaluate PUMiner on more than 17.2 million posts on Stack Overflow and 52,611 posts on Security StackExchange. We show that PUMiner is effective with the validation performance of at least 0.85 across all model configurations. Moreover, Matthews Correlation Coefficient (MCC) of PUMiner is 0.906, 0.534 and 0.084 points higher than one-class SVM, positive-similarity filtering, and one-stage PU models on unseen testing posts, respectively. PUMiner also performs well with an MCC of 0.745 for scenarios where string matching totally fails. Even when the ratio of the labelled positive posts to the un-labelled ones is only 1:100, PUMiner still achieves a strong MCC of 0.65, which is 160% better than fully-supervised learning. Using PUMiner, we provide the largest and up-to-date security content on Q&A websites for practitioners and researchers. |
Keywords: | Mining Software Repositories; Positive Unlabelled Learning; Machine Learning; Natural Language Processing; Software Security |
Description: | Co-located with ICSE '20: 42nd International Conference on Software Engineering. |
Rights: | © 2020 Association for Computing Machinery. |
DOI: | 10.1145/3379597.3387443 |
Published version: | https://dl.acm.org/doi/proceedings/10.1145/3379597 |
Appears in Collections: | Aurora harvest 4 Computer Science publications |
Files in This Item:
There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.