[Update: see also a later post.]
We unveiled this week the first release of Swift, a system for the specification, execution, and management of applications comprising many tasks coupled by disk-resident datasets. Such applications are common when analyzing large quantities of data, performing parameter studies, and/or executing ensemble simulations. (The word "workflow" is often used for such applications, but it doesn't sound right to me) The open source Swift software combines:
- A simple scripting language, SwiftScript, for the concise, high-level specification of such computations (without regard to data layout or location), and
- An execution engine for the rapid and reliable dispatch of many tasks to many processors, whether on parallel computers, campus grids, or multi-site grids.
Swift users in the physical, biological, and social sciences; the humanities; computer science; and education have achieved multiple-order-of-magnitude savings (!) in program development and execution time, relative to approaches based on shell scripts and other ad hoc technologies.
Swift builds on work performed with National Science Foundation's Grid Physics Network (GriPhyN) project on the Virtual Data System (VDS). Work on another VDS component, Pegasus, continues at USC/ISI.
The Swift team comprises Yong Zhao (imminent PhD, already interviewing), Mike Wilde, Mihael Hatigan, Tibi Stef-Praun, Ben Clifford, and Nika Nefedova. Gregor von Laszewski architected (and Mihael Hatigan built) the Karajan runtime system on which Swift is built. Globus services are used to access remote computers and to move data.
Comments