skypilot-users
How can I trigger an event to pass data from Blob storage to a running LLM for text summarization?
I want to build a system where I have text data stored in the form of key-value pairs on Blob storage. I want to read this data and pass it to a running LLM for text summarization. How can I trigger an event whenever there is new data on Blob storage and pass it to the LLM?
Su
Sumit Sharma
Asked on Feb 08, 2024
There are two ways you can achieve this:
-
If the key-value pairs appear frequently, you can have a GPU cluster always up and pulling from the Blob storage. The worker can write a file on the Blob storage to coordinate with other workers in a multi-node case.
-
If the key-value pairs appear less frequently, you can have a CPU instance doing the pulling and create GPU clusters on-demand.
Feb 08, 2024Edited by