Please use this identifier to cite or link to this item: https://hdl.handle.net/2440/129195
Citations
Scopus Web of Science® Altmetric
?
?
Type: Conference paper
Title: PUMiner: Mining security posts from developer question and answer websites with PU learning
Author: Le, T.H.M.
Hin, D.
Croft, R.
Babar, M.A.
Citation: IEEE International Working Conference on Mining Software Repositories, 2020, pp.350-361
Publisher: ACM
Publisher Place: New York, NY
Issue Date: 2020
Series/Report no.: IEEE International Working Conference on Mining Software Repositories
ISBN: 9781450375177
ISSN: 2160-1852
Conference Name: 17th International Conference on Mining Software Repositories (MSR) (29 Jun 2020 - 30 Jun 2020 : Virtual)
Statement of
Responsibility: 
Triet Huynh Minh Le, David Hin, Roland Croft, and M. Ali Babar
Abstract: Security is an increasing concern in software development. Developer Question and Answer (Q&A) websites provide a large amount of security discussion. Existing studies have used human-defined rules to mine security discussions, but these works still miss many posts, which may lead to an incomplete analysis of the security practices reported on Q&A websites. Traditional supervised Machine Learning methods can automate the mining process; however, the required negative (non-security) class is too expensive to obtain. We propose a novel learning framework, PUMiner, to automatically mine security posts from Q&A websites. PUMiner builds a context-aware embedding model to extract features of the posts, and then develops a two-stage PU model to identify security content using the labelled Positive and Un-labelled posts. We evaluate PUMiner on more than 17.2 million posts on Stack Overflow and 52,611 posts on Security StackExchange. We show that PUMiner is effective with the validation performance of at least 0.85 across all model configurations. Moreover, Matthews Correlation Coefficient (MCC) of PUMiner is 0.906, 0.534 and 0.084 points higher than one-class SVM, positive-similarity filtering, and one-stage PU models on unseen testing posts, respectively. PUMiner also performs well with an MCC of 0.745 for scenarios where string matching totally fails. Even when the ratio of the labelled positive posts to the un-labelled ones is only 1:100, PUMiner still achieves a strong MCC of 0.65, which is 160% better than fully-supervised learning. Using PUMiner, we provide the largest and up-to-date security content on Q&A websites for practitioners and researchers.
Keywords: Mining Software Repositories; Positive Unlabelled Learning; Machine Learning; Natural Language Processing; Software Security
Description: Co-located with ICSE '20: 42nd International Conference on Software Engineering.
Rights: © 2020 Association for Computing Machinery.
DOI: 10.1145/3379597.3387443
Published version: https://dl.acm.org/doi/proceedings/10.1145/3379597
Appears in Collections:Aurora harvest 4
Computer Science publications

Files in This Item:
There are no files associated with this item.


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.