SkyPilot - Is it possible to run a distributed job on VMs in different regions?

skypilot-users

Is it possible to run a distributed job on VMs in different regions?

Nikhil is asking if it's possible to run a distributed job on many VMs in different regions while using SkyPilot for the job. Zongheng Yang from SkyPilot is discussing the use case and suggesting possible solutions.

Ni

Nikhil

Asked on Mar 05, 2024

Yes, it is possible to run a distributed job on VMs in different regions.
One approach is to create multiple clusters, each in a different region.
For batched inference tasks across a large dataset on multiple nodes, you can use a for-loop and commands like sky launch --down or sky spot launch to achieve the desired distribution.

Mar 05, 2024Edited by