Alex Kim
Asked on Sep 27, 2023
Yes, there is potential for supporting Databricks. Databricks is good at running data-intensive jobs, but there is a need for running compute-intensive tasks on small/medium-sized datasets. Integrating SkyPilot with Databricks can provide the solution for this. It is already possible to run Ray on Databricks, which opens up the possibility for further integration.