HTC-04: Run a set of interrelated jobs on several HTC resources

Executive Summary:

A researcher needs to run one or more applications many times with varying inputs on several community high-throughput computing (HTC) resources. Each application run is called a “job.” The complete set of jobs is called a “project.” Some jobs may use the outputs of other jobs as their inputs. Some jobs may need to run on the same resource as other jobs, possibly at the same time. (These are called “dependencies.”) Once the jobs and dependencies are specified to the system, the system executes them without further involvement by the researcher.

User Importance Summary:

Some research projects require a large sequence or pattern of jobs to be executed on one or more HTC resources. The human labor to manage individual jobs is a poor use of time for highly skilled researchers. A workflow system that automates job submission and management improves research productivity.

Target Communities and Sizes:

Researchers - 1,000 < N < 10,000 Educators who use community resources - 100 < N < 1,000 Science gateway developers - 100 < N < 1,000

Documents:

Full description (public)
Full description (for review)
Full description (drafting version)
Capability delivery plan

Status:

Approved

Use Case End User:

researchers

educators

science gateway developers

Area:

Scientific Computing