An Efficient and Reliable Scientific Workflow System

Tulio Tavares; George Teodoro; Tahsin Kurc; Renato Ferreira; Dorgival Guedes; Wagner Meira Jr.; Umit Catalyurek; Shannon Hastings; Scott Oster; Steve Langella; Joel Saltz

doi:10.1109/CCGRID.2007.20

Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid '07)

An Efficient and Reliable Scientific Workflow System

Year: 2007, Pages: 445-452

DOI Bookmark: 10.1109/CCGRID.2007.20

Authors

Tulio Tavares, Department of Computer Science, Universidade Federal de Minas, Brazil
George Teodoro, Department of Computer Science, Universidade Federal de Minas, Brazil
Tahsin Kurc, Department of Biomedical Informatics, Ohio State University, USA
Renato Ferreira, Department of Computer Science, Universidade Federal de Minas, Brazil
Dorgival Guedes, Department of Computer Science, Universidade Federal de Minas, Brazil
Wagner Meira Jr., Department of Computer Science, Universidade Federal de Minas, Brazil
Umit Catalyurek, Department of Biomedical Informatics, Ohio State University, USA
Shannon Hastings, Department of Biomedical Informatics, Ohio State University, USA
Scott Oster, Department of Biomedical Informatics, Ohio State University, USA
Steve Langella, Department of Biomedical Informatics, Ohio State University, USA
Joel Saltz, Department of Biomedical Informatics, Ohio State University, USA

Abstract

This paper presents a fault tolerance framework for applications that process data using a distributed network of user-defined operations in a pipelined fashion. The framework saves intermediate results and messages exchanged among application components in a distributed data management system to facilitate quick recovery from failures. The experimental results show that the framework scales well and our approach introduces very little overhead to application execution.

Like what you’re reading?

Already a member?

Get this article FREE with a new membership!

Using Runtime Verification to Design a Reliable Execution Framework for Scientific Workflows
2009 Sixth IEEE Conference and Workshops on Engineering of Autonomic and Autonomous Systems
Towards Security Hardening of Scientific Demand-Driven and Pipelined Distributed Computing Systems
2008 International Symposium on Parallel and Distributed Computing
Reliable Distributed Real-Time and Embedded Systems through Safe Middleware Adaptation
2012 IEEE 31st International Symposium on Reliable Distributed Systems (SRDS 2012)
Building Reliable Data Pipelines for Managing Community Data Using Scientific Workflows
2009 5th IEEE International Conference on e-Science (e-Science 2009)
Client/Server Qualities: A Basis for Reliable Distributed Workflow Management Systems
Proceedings of 17th International Conference on Distributed Computing Systems
A Taxonomy for the Analysis of Scientific Workflow Faults
2010 13th IEEE International Conference on Computational Science and Engineering
On Creating Efficient Object-relational Views of Scientific Datasets
2006 International Conference on Parallel Processing (ICPP'06)
Medical image processing workflow support on the EGEE grid with taverna
2009 22nd IEEE International Symposium on Computer-Based Medical Systems
Data-Preservation in Scientific Workflow Middleware
18th International Conference on Scientific and Statistical Database Management
Efficient Fault Tolerance for Pipelined Query Engines via Write-ahead Lineage
2024 IEEE 40th International Conference on Data Engineering (ICDE)

An Efficient and Reliable Scientific Workflow System

Authors

Abstract

Related Articles