Job Engine jobs often consist of several phases, with each proceeding in a predefined sequence. Such jobs run the gamut from those jobs that have a single phase, such as TreeDelete, to complex jobs, such as FlexProtect and MediaScan, which have multiple distinct phases.

A job phase must be completed in its entirety before the job can progress to the next phase. If any errors occur during processing, the job is marked “failed” at the end of that particular phase, and the job is terminated.

Each job phase has multiple work chunks, or tasks. Tasks, which have multiple individual work items, are divided up and load balanced across the nodes within the cluster. Successful completion of a work item produces an item result, which might contain a count of the number of retries required to repair a file, plus any errors that occurred during processing.

Figure 2.   OneFS Job Engine job hierarchy