skypilot-users
How can I upload files from a local directory to an existing S3 bucket using the Python API?
Sh
Shreehari Vaasistha L
Asked on Nov 20, 2023
To upload files from a local directory to an existing S3 bucket using the Python API, you can use the sky
library. However, note that the library currently does not support mounting a specific sub-path on a bucket. The root of the bucket is mounted at the specified mount point. To achieve your goal, you can change the path where you write your checkpoints such that it is relative to the root of the mount point. Here's an example snippet:
import contextlib
import sky
cluster = "my-cluster"
with contextlib.suppress(ValueError):
sky.down(cluster)
# Create a task
# Set resources
# Set storage mounts
# Launch the task
Dec 19, 2023Edited by