Ian Foster

Virtual Clusters: A Couple of Developments

Tim Freeman writes about how to create one click clusters using the Globus workspace service.

Univa UD describes how to create UniCluster clusters on EC2.

We wrote a paper about on-demand clusters a while back. It's how all clusters should be created :)

GridFTP profiled (and nice picture of Raj)

iGSTW has a nice article on the marvel that is GridFTP (and a nice photo of Raj Kettimuthu). The article says "these features will be in GT4.2, to be released in summer 2008." GT4.2 was released on July 2.

Globus Toolkit 4.2 Released!

Charles Bacon, GT integration lead, writes: "On behalf of the Globus Toolkit development team I am pleased to announce that a new stable release of the Globus Toolkit is now available.  GT4.2.0 contains an upgrade to the web services specifications used by the toolkit [to the final WSRF and WS-Notification specifications] as well as new features in all services.  New users are encouraged to use the 4.2.0 release.  Existing users may wish to evaluate the new software while maintaining their existing installations;  due to the specification upgrade, the web services are incompatible with the 4.0.x series.  Details on the spec upgrade are available in the release notes.

Relevant 4.2.0 links:
- Release notes: http://www.globus.org/toolkit/docs/4.2/4.2.0/rn/release_notes.html
- Software: http://www.globus.org/toolkit/downloads/4.2.0/
- Documentation: http://www.globus.org/toolkit/docs/4.2/4.2.0/

Thanks for your support of Globus software!"

SAML-Based Attributes for Globus Grids

The GridShib Project announced last week the release of GridShib for Globus Toolkit v0.6.0.  This is an exciting development, as GridShib software allows for powerful new authorization architectures in which access control decisions are made based on attributes obtained from many different sources. From their announcement:

This  release culminates a 20-month effort to bring SAML-based attribute push to X.509-based Grids.

GridShib for Globus Toolkit (GT) is an implementation of a Grid Service Provider, an entity much like a SAML Service Provider but for Grids.  A Grid Service Provider consumes X.509-bound SAML tokens, a new type of security token that enables attributed-based authorization in X.509-based Grids.

Most everything you need to know about GridShib for GT is on this web page:

http://gridshib.globus.org/docs/gridshib-gt-0.6.0/readme.html

On this readme page, you will find more detailed information about the GridShib for GT software as well as links to downloads and documentation.

A major advance in this version of GridShib for GT is support for the TeraGrid Science Gateway use case where an intermediary makes a grid request on behalf of a browser user.  The Gateway binds a SAML token to an X.509 proxy certificate and makes a request to a
gridshib-enabled web service.  On the service side, GridShib for GT consumes the SAML token and makes an access control decision based on the security information in the token.

As a SAML-consuming software component, GridShib for GT complements the previously released GridShib SAML Tools and GridShib Certification Authority (CA), which are SAML-producing software components.  These three components together enable attribute-based authorization in X.509-based Grids.  See the Quick Start for step-by-step instructions that show how to use GridShib for GT v0.6, GridShib SAML Tools v0.3, and GridShib CA v0.5.1 together on Windows and UNIX systems:

http://gridshib.globus.org/docs/gridshib/quick-start.html

For links to all GridShib software downloads and additional documentation, visit the GridShib Downloads page:

http://gridshib.globus.org/download.html

Funding for GridShib software has been provided by the NSF NMI program and the NSF TeraGrid program.

10 Reasons to Attend Open Source Grid and Cluster Conference

Ok, I admit it is corny--but I assembled a list of 10 reasons why you should attend the Open Source Grid and Cluster Conference, to be held in Oakland May 12-16 (www.opensourcegridcluster.org).

1) Globus program is fantastic, including tutorials, advanced technical presentations, contributed talks, and community events on every aspect of Globus.

2) Gobs of other material on Sun Grid Engine and Rocks, and other open source grid and cluster software.

3) Gathering: A great opportunity to meet colleagues, peers, collaborators from the grid and cluster community. The only grid meeting in the US the rest of this year--the next two OGFs are in Spain (June) and Singapore (September).

4) GT4.2: You'll get to learn about the exciting new features in Globus Toolkit 4.2. New execution, data, security, information, virtualization, and core services.

5) Gratfication (immediate) as you get to provide your input on future directions for Globus, Sun Grid Engine, Rocks, and other open source systems--and maybe sign up to contribute to those developments.

6) Grid solutions: You'll get to meet the people using Globus to build enterprise grid solutions in projects like caBIG, TeraGrid, Earth System Grid, MEDICUS, and LIGO, and learn about solution tools like Introduce, MPI-G, Swift, Taverna, and UniCluster.

7) Gurus: You get to grill the Globus gurus--or, if you prefer, show off your own Globus guru status.

8) Great price: $490 registration is substantially cheaper than OGF or HPDC, for example, and the hotel rate is reasonable ($149).

9) Gorgeous location: Oakland is easy to get to -- SFO (with easy BART  train ride), Oakland, and San Jose airports also nearby. Just a 10 minute train ride to download San Francisco. A lovely time to be in the Bay Area.

10) Gorilla and guerilla free: None of the corporate marketing talks that diluted the last GridWorld conference--apart from two sponsor talks, this is pure tech, and highly useful tech at that.

We look forward to seeing you in Oakland!

Regards -- Ian.

Clouds over Chicago

Way before clouds were popular (remember then?) my colleagues Kate Keahey and Tim Freeman started work on their workspace service, a system for on-demand creation and management of virtual machines on remote computing systems. They now have an implementation that interfaces both to clusters running conventional schedulers and to Amazon EC2. It's distributed as part of the Globus software, or you can download it separately.

Kate and Tim have recently established a deployment of the workspace service in the Computation Institute at U.Chicago and Argonne. With a nod to the new cloud meme, they've named it Nimbus. They say:

The University of Chicago Science Cloud, codenamed "Nimbus", is a web service that delivers compute capacity in the cloud for scientific communities. The Nimbus' simple client allows you to obtain customized compute nodes (that we call "workspaces") that you have full control over quickly, easily, and in ways that can be fully automated. Using the Nimbus cloud you can request the exact compute capability you currently need for your application and scale it up or down as your needs dictate.

Nimbus provides compute capability in the form of Xen virtual machines (VMs) that are deployed on physical nodes of the University of Chicago TeraPort cluster using the workspace service. We currently make 16 nodes of the TeraPort cluster available for cloud computing. Nimbus is available for members of scientific community wanting to run in the cloud. To obtain access you will need to provide a justification (a few sentences explaining your science project) and a valid grid credential (If you don't have a credential, email us. We can help). Based on the project, you will be given an allocation on the cloud. Send your requests, demands and cries of anguish to workspace-user@globus.org (for cries of anguish mp3 format is acceptable).

In a typical session you will make a request to deploy a workspace based on a specified VM image. You can either use one of the VM images already available on the cloud (we provide a command that allows you to see what's already there) or upload your own VM image. On deployment, the image will be configured with an ssh public key you provide -- in this way once the workspace is deployed, you will be able to ssh into it and configure it further, upload data, or run your applications. Have fun!

Services for Science

I gave a talk on Services for Science (PDF, PPT) at the INGRID 2008 conference in Ischia, Italy, on Wednesday. I decided to do something different and include demonstrations. I think this worked well. I created and deployed a GT4 service using Introduce and gRAVI, and then created and ran a workflow invoking GT4 services using Taverna.  Well, to be honest, there was some steps I skipped along the way (in the style of Julia Childs), but nevertheless I found it impressive how much could be done interactively, in a short time. In summary, I was able to show how we can:

1. Create a new service using the Introduce integrated development environment, defining operations and resource properties, folding in required functionality (e.g., security, notification), and selecting types from both base types and predefined libraries. (Using gRAVI we can also encapsulate executables.)
2. Publish this service into registries (GT4 index services).
3. Discover available services.
4. Compose services into workflows, e.g., via the use of Taverna, which thanks to recent work by Wei Tan and Ravi Madduri (and much help from the Taverna team) can now invoke GT4 services.
5. Deploy and publish the workflow in turn ...

Grid Summer School

The sixth in the highly successful series of International Summer  Schools on Grid Computing will be held at the Hotel Füred Conference and Congress Centre of Balatonfüred, Hungary, from 6th to 18th July 2008.

The School will include lectures, discussions, laboratory sessions, tutorials and group work delivered by leading authorities in the fields of advanced grid technology, applications of e-Science and distributed systems research. Reports from world leaders in deploying and exploiting Grids will complement lectures from research leaders shaping future e-Infrastructure.

Hands-on laboratory exercises will give students experience with widely used Grid middleware. The school will conclude with an integrating practical that will enable students, working in teams, to bring together all they have learnt on an extended exercise that simulates collaborative research using e-Infrastructures. Indeed during the school, participants will meet like-minded students from many parts of the world, working in many disciplines, and form valuable long-term working relationships.

We invite applications from enthusiastic and ambitious researchers who have recently started or are about to start working on Grid projects. Students may come from any country. We expect participants from computer science, computational science and any application discipline. The School will assume that students have diverse backgrounds and build on that diversity. However, in order to fully participate in the practical exercises you should be a confident programmer who will have fulfilled certain prerequisites.

To find further details visit the web site at: http://www.issgc.org

Nice article on caBIG cancer biomedical informatics grid

I keep mentioning caBIG. But here is a nice profile in ComputerWorld that describes the project's goals and status.

New Globus release -- big GRAM improvements

Charles Bacon, Globus release manager, writes: "On behalf of the Globus Toolkit development team I am pleased to announce that a new incremental release of GT4 is now available for download. GT4.0.7 is recommended for all users.It was released because of bug 5910, a potential RFT data corruption bug.  The bug affected only GT4.0.6, and users of GT4.0.6 can apply the update package from http://www.globus.org/toolkit/advisories.html.  New users are encouraged to start with the 4.0.7 release, as other bugs were also fixed as listed in the release notes.

Relevant 4.0.7 links:
- Release notes: http://www.globus.org/toolkit/releasenotes/4.0.7/
- Software: http://www.globus.org/toolkit/downloads/4.0.7/
- Documentation: http://www.globus.org/toolkit/docs/4.0/

Thanks for your support of Globus software!"

Among other things, GT4.0.7 includes further significant improvements to GRAM performance and scalability.