SkyPilot - Is Speculative Decoding coming to skypilot?

SkyPilot

skypilot-org.slack.com

powered by

Snowshoe.DEV

skypilot-users

Is Speculative Decoding coming to skypilot?

I'm wondering if Speculative Decoding will be added to skypilot. Can skypilot run any library/framework that supports speculative decoding?

Sas Sun

Asked on Jan 23, 2024

Yes, skypilot should be able to run any library/framework that supports speculative decoding. Do you have a particular one in mind?

Jan 25, 2024Edited by