skypilot-users

Is there a plan for supporting Databricks?

Al

Alex Kim

Asked on Sep 27, 2023

Yes, there is potential for supporting Databricks. Databricks is good at running data-intensive jobs, but there is a need for running compute-intensive tasks on small/medium-sized datasets. Integrating SkyPilot with Databricks can provide the solution for this. It is already possible to run Ray on Databricks, which opens up the possibility for further integration.

Sep 27, 2023Edited by