Swift in Munich

I visited LRZ in Munich a few weeks ago. As well as meeting the excellent team there working on Globus for D-Grid and related projects, I gave a talk on our recent work with Swift. (See earlier post.) The talk includes results that my colleague Yong Zhao we will present at the IEEE Workshop on Scientific Workflows in July, and also some results on dynamic provisioning that Ioan Raicu obtained for a recent SC submission. I'm excited about how this work is going--we're running increasingly large computations for an increasing number of applications.

The abstract for the talk follows:

A common pattern in scientific computing involves the execution of many tasks that are coupled only in the sense that the output of one may be passed as input to one or more others--for example, as a file, or via a Web Services invocation. While such “loosely coupled” computations can involve large amounts of computation and communication, the concerns of the programmer tend to be different than in traditional high performance computing, being focused on management issues relating to the large numbers of datasets and tasks--and often, the complexities inherent in “messy” data organizations--rather than the optimization of interprocessor communication. To address these concerns, we have developed Swift, a system that combines a novel scripting language called SwiftScript with a powerful runtime system based on Karajan and Falkon to allow for the concise specification, and reliable and efficient execution, of large loosely coupled computations. Swift adopts and adapts ideas first explored in the GriPhyN virtual data system, improving on that system in many regards. I describe the SwiftScript language and its use of XML Data Typing and Mapping notation to describe the logical structure of complex file system structures. I also present the Swift system and its use of Karajan, Falkon, and Globus services to dispatch and manage the execution of many tasks in different execution environments. I summarize application experiences and detail performance experiments that quantify the cost of Swift operations.

Comments

[Shao Baolin]

I am a Chinese student and just starting to work on Grid Computing. And I find a lot of useful materials from your homepage. Thank you very much.

By the way, I have never seen any prof. writting his or her blog, haha, interesting!