A methodology for predictive topic modelling; or, any excuse to watch Love Actually

Abstract

Topic modelling is an area of natural language processing (NLP) in which a corpus of text documents is summarised by an underlying structure of `topics', or themes. Due to the incredibly complex nature of human language, we often require ways to meaningfully summarise the information contained in a piece of text. Topic models provide a method in which we are able to keep substantial semantic information, but still work with a small number of variables. Topic modelling has mainly been applied to machine learning problems, with little emphasis on prediction from text. This thesis provides a statistical framework for prediction from, or about, text, using topic models as a data reduction method and the topics themselves as predictors. The results of this thesis show that while using individual words as predictors in a regression model remains the most accurate method, it is far too computationally expensive to apply to large corpora. However, the topic regression models proposed here perform comparably, and at a much lower computational cost. We also show that incorporating more information, such as the structure of language, into topic model inference improves the predictive capability of the topics. This thesis therefore proposes a computationally viable, well-performing method for prediction from text. From here, we may consider adapting additional topic models to a regression framework, depending on the problem at hand and its requirements. These methods, while tested in this thesis on relatively small corpora, would also be applicable to big data problems.