created by bshifaw
on 2018-04-11
Preemptibles are often used in featured workflows and can have large effects on cost and time of a submitted workflow. When preemptibles are enabled for a task, Google dramatically reduces the cost of running the task. These tasks may be terminated (preempted) if Google’s Compute Engine decides it needs that VM instance for other tasks. Cromwell will in turn attempt to retry the task on a different machine, but this will increase the time in completing the task and executing the overall workflow.
Let’s say for example User A has a workflow running with preemptibles enabled and User B is running a workflow but decides to disable preemptibles to have their submission complete as quickly as possible. Since preemptibles are disabled for User B, they can be considered as having the ability to preempt another machine. If Google is running low on compute resources it may stop the task User A is running and allow User B to run their workflow first. Cromwell will retry running User A task on another VM resource to continue the workflow but it means restarting a task. This may slow User A overall execution time but may save them a considerable amount on cost.
It is possible to decide the number of times a tasks within a workflow will be preempted before running the task on a full-cost non-preemptible machine. Not using the preemption feature or setting preemption to zero would mean the workflow would never get preempted, preemption of 1 would mean a user’s tasks would be preempted at most once, preemption of 2 would mean a user’s tasks would be preempted at most twice, and so on. Preemptibles provide the user with a gradient level of choice to decide whether their focus is on saving time or saving money.
Further details on Gcloud Preemptibles is described [here](https://cloud.google.com/compute/docs/instances/preemptible).
Updated on 2018-04-11