Data transfer and sharing within web service workflows.

Abstract

With the development of distributed systems, it is more and more common for users to harness different resources to implement a larger task to meet their requirements. Among the different approaches to distributed resource coordination, workflows based on Service Oriented Architecture (SOA) is an important case, as SOA provides a framework that is designed for loosely coupled applications. This thesis introduces the research work that we have carried out in distributed computing environments to improve the performance of data transfer and sharing in a web service workflow. In a distributed environment, we explore how to improve the efficiency of data moving between services in a web service workflow. Data movement in a web service workflow can be categorized into two classes: data transfer between any two nodes in the workflow and the intermediate data sharing between different web services. We initially explored ways to improve the data transfer performance between two web service nodes, then improved the data sharing performance via study of the data sharing relationship between applications composed in a workflow. We carried out the initial part of the research work based on the CIMA (Common Instrument Middleware Architecture) web service interface, which has been used by multiple academic organizations as an interface to distributed scientific instruments and applications. With the related experiments, we explore how the data generated by instruments can be transferred efficiently between different web service nodes. In the rest of the research, we study the data sharing relationships between different web service applications. By proposing the Web Service Data Forwarding (WSDF) framework, we allow intermediate data to be forwarded directly from the data generator to its consumer without going via a third party (the workflow engine). We have implemented prototype systems for our proposed ideas. We also tested these systems in different environments to demonstrate the performance improvement that is expected from the WSDF approach.