Peer-to-Peer data sharing for scientific workflows on amazon EC2

Abstract

—In this paper, we consider the problem of data sharing in scientific workflows running on the cloud. We present the design and evaluation of a peer-to-peer approach to help solve this problem. We compare the performance of our peer-to-peer file manager with that of two network file systems for storing data for a typical data-intensive workflow application. Our results show that while our peer-to-peer file manager performs significantly better than one of the network file systems tested, it does not perform as well as the other. Finally, we discuss the various issues that might have affected the performance of our peer-to-peer file manager.