Ian Foster

Visit Florida in January (and learn about grid)

Florida International Grid School (FIGS'08)  --  Apply by Dec 10 !

JOIN US for an exciting 3-day course in large-scale and high-performance grid computing to take place January 23-25, 2008, at Florida International University, Miami, FL.

This intensive course introduces the techniques of grid and distributed computing for science and engineering fields, with hands-on training in the use of large-scale grid computing resources. The course introduces skills that will be needed by researchers in the natural and applied sciences, engineering, and computer science to conduct and support large-scale computation and data analysis in emerging grid and distributed computing environments. The workshop will focus on enabling the use of Open Science Grid (OSG) and TeraGrid cyberinfrastructure to perform large-scale computations and data-intensive processing in various research fields. Participants will learn how to use grids of thousands of processors and will be able to continue to use these resources for their research after the course.

The workshop will cover:

* Overview of distributed computing concepts and tools
* Wide-area high speed optical networking
* Concepts, tools, and techniques of grid computing
* Discovering and using grid resources
* Grid scheduling and distributed data management
* Web service and grid service concepts
* Techniques for workflow and collaboration

Undergraduate and graduate students, researchers, educators and professionals in engineering, computer science, or any scientific, data-or computing-intensive discipline may apply. Applicants should have at least intermediate programming skills (one to two semesters experience in C/C++, Java, Perl, and/or Python) and hands-on experience with UNIX / Linux in a networked environment.

Important Dates:

Application Deadline: Dec 10.
Notification of Acceptance will be sent by Dec 20.
Registration Deadline: Jan 10.

Application form can be found at: www.ci.uchicago.edu/figs08

For information on FIGS'08 please visit www.opensciencegrid.org/workshop or
send a message to figs08@opensciencegrid.org

McKinsey Report on Emissions Reduction

A report released by McKinsey, Reducing U.S. Greenhouse Gas Emissions: How Much at What Cost?, concludes that the United States could cut its projected 2030 greenhouse gas emissions by 33% to 50% at modest cost, using energy efficiency and other proven, on-the-shelf technologies. As much as 40 percent of the emission reductions could result from policy measures that more than pay for themselves over their lifetimes.

In one of the report's more notable findings, aggressive application of energy efficiency technology in new and existing homes and commercial buildings could eliminate the need for 80 percent of new electricity generation that would be required to meet new demand in 2030 under a business-as-usual scenario in which no new efficiency improvements are made.

Interestingly, of the many emission reduction methods considered, hybrid cars are by far the most expensive.

MapReduce info

Rick Stevens pointed me at this post describing Yahoo!'s work with Hadoop, the open source implementation of MapReduce. Of course, that is old news, but some navigation led me to this nice MapReduce cookbook for machine learning.

Many many years ago I wrote a book on parallel programming. My initial plan was to show how just about everything could be expressed using map reduce, but I couldn't quite see how to package things in a way  way that made sense. (Probably I wanted to do it all via language constructs.) So instead I ended up covering a lot of other material instead. It's neat to see how with the right abstractions things turn out so nicely.

A Plan for the Improvement of English Spelling

by Mark Twain

For example, in Year 1 that useless letter "c" would be dropped to be replased either by "k" or "s", and likewise "x" would no longer be part of the alphabet. The only kase in which "c" would be retained would be the "ch" formation, which will be dealt with later. Year 2 might reform "w" spelling, so that "which" and "one" would take the same konsonant, wile Year 3 might well abolish "y" replasing it with "i" and Iear 4 might fiks the "g/j" anomali wonse and for all.         

Jenerally, then, the improvement would kontinue iear bai iear with Iear 5 doing awai with useless double konsonants, and Iears 6-12 or so modifaiing vowlz and the rimeining voist and unvoist konsonants. Bai Iear 15 or sou, it wud fainali bi posibl tu meik ius ov thi ridandant letez "c", "y" and "x" -- bai now jast a memori in the maindz ov ould doderez -- tu riplais "ch", "sh", and "th" rispektivli.

Fainali, xen, aafte sam 20 iers ov orxogrefkl riform, wi wud hev a lojikl, kohirnt speling in ius xrewawt xe Ingliy-spiking werld.

(Thanks to Ido Rosen for this.)

Workshop on Building Effective Virtual Organizations

I am involved in organizing  an NSF-sponsored  workshop, "Building Effective Virtual Originations" (BEVO) to be held in Washington, DC, on January 15 and 16. As we write on the Web page:

Virtual organizations are increasingly central to the science and engineering projects funded by the National Science Foundation. Indeed, if you are a researcher or educator, chances are that you if don't already lead or participate in at least one distributed team, you will soon.

Unfortunately, chances are also that you have never been told how to establish such teams, how to make them successful, or what technologies exist that can help them function effectively. You probably haven't had many opportunities to interact with others building, or just working in, virtual organizations, either.

Our goal at this workshop is to help address this knowledge gap. We invite you to take this opportunity to come and learn what is required to make virtual organizations successful, contribute your experiences and challenges to the discussions, and to establish new connections that will help you succeed in your research and education projects in the future.

The workshop is sponsored by the National Science Foundation's Office of Cyberinfrastructure, which has identified virtual organizations as a fundamental element of its infrastructure plans. Attendees will be responsible for their own travel and hotel. There is no fee to attend the workshop, but registration is required as space is limited.

We're still finalizing some of the speakers, but this is a unique opportunity both to learn about what it means to make virtual organizations work in practice, and to meet and talk with others engaged in the "virtual organization lifestyle" ...

On Interdisciplinary Work

I recently came across this quote from Théophile Gautier (1811–1872)'s poem L'Art:

Oui, l'oeuvre sort plus belle
D'une forme au travail
Rebelle,
Vers, marbre, onyx, émail.

[Yes, the work comes out more beautiful from a material that resists the process, verse, marble, onyx, or enamel (not email!)]

I think this summarizes well the merits of interdisciplinary work and the challenges and joys of system-level science.

Earth to be paradise; distance to lose enchantment

I recently had the occasion to present a talk on "virtual environments and knowledge production." This got me thinking about just how virtual environments can help with the important tasks of enhancing both individual and collective human creativity. My talk tries to highlight some of what has been accomplished and some of the challenges that lie ahead. (However, as I continue to strive to remove words from slides [recalling Beckett's admonition that "every word is an unnecessary stain on silence and nothingness"], the slides themselves may not make much sense!) I mentioned the following as some of issues that must be addressed for virtual worlds (a subset of virtual environments) to be useful for "knowledge production":

• Integration with the physical world, e.g., sensors and instrumentation
• Integration with the rest of the cyberworld
• Integration with simulation
• Security and trust, in their many forms
• Abstractions, metaphors, interfaces
• Scale (data volumes, simulation fidelity)

During my reading, I came across a nice article by Gary Olson and Judith Olson, Distance Matters, which included a delightful quote from Arthur Mee:

If, as it is said to be not unlikely in the near future, the principle of sight is applied to the telephone as well as that of sound, earth will be in truth a paradise, and  distance will lose its enchantment by being abolished altogether.

(Arthur Mee was best known for his Children's Encylopedia, but here he is writing in 1898 about the impact of the telephone.)

Grid.org

Univa UD has re-launched "grid.org"--previously dedicated to philanthropic computing projects--as:

a single location where open-source cluster and grid information can be aggregated and where community members can exchange information, experiences, and ideas related to the complete open source cluster software stack. In particular, but not exclusively, Grid.org provides a community where users of Cluster Express, and the various open source components it comprises, can interact with each other and with the source code.

I recall finding it frustrating that the URL "grid.org" (which I would have loved to have back when our Grid book first came out) was devoted to "desktop computing." But things sometimes come to those who wait!

For those interested in applying their home computers to philanthropic computing projects, I recommend the World Community Grid, which harnesses idle computers for useful purposes such as modeling potential AIDS and cancer drugs.

HealthGrid Comes to Chicago, June 2-4 2008

The opportunities to apply grid  computing methods in health care are, simply put, enormous. (Irving Wladawsky-Berger refers to it as the "ASCI of Grid" to imply that the challenges are comparable in their extreme scale to those tackled by the DOE ASCI program in simulation. That is an understatement.) There is an urgent need for community, best practices, standards, and the like.

These considerations motivated the formation of the HealthGrid.US Alliance (HG.US), a partnership of scientific, medical and technology professionals from academia, industry and government, whose shared mission is to promote the application of advanced information technology to solve cutting-edge problems in Biomedical Science and Healthcare. HG.US is an affiliate of the international HealthGrid Association.

As a first action, HG.US is sponsoring the first HealthGrid Annual Meeting to be held outside of Europe, in Chicago, Illinois, USA, June 2-4 2008. See the announcement (pdf). The previous five meetings (2003-2007, held in Europe, have formal published proceedings that are also available from the website.

Relative debugging with Guard

When modifying software, or ported it from one system to another, one often ends up checking for correctness (and attempting to diagnose errors) by comparing the execution of a new program with a reference version that is known to work. My Australian friend and colleague David Abramson developed a tool called Guard to automate this process. With Guard, you specify the variables you want to monitor, and when, and then fire up the new and reference versions of your software. Guard monitors the specified variables, and notifies you when their values differ.

John Michalakes and I had the occasion to work with  Guard back in 1996 when we applied it in a project developing a parallel mesoscale weather model, MM5. It was spookily wonderful to see differences between the parallel and sequential implementations become visible in real-time in a 3-D visualization. (The figure shows a 2-D plot of differences, which is also useful but less beautiful.) It also then became extremely easy to fix those problems. We wrote a paper together on this work, which won a best paper award at SC'96, due I think to David's presentation skills--and the videos.)

This technology has now been licensed by Cray for use in their new Cascade program. It's exciting to see the technology making its way into mainstream use.