Fine-Tuning Large Language Models at Scale - Workday's Approach

Discover Workday's innovative approach to fine-tuning Large Language Models (LLMs) at scale in this Ray Summit 2024 presentation. Explore how Trevor DiMartino addresses the challenges of training models on isolated customer data within a secure, multi-tenant environment while managing GPU scarcity and strict data access controls. Learn about Workday's platform design, which utilizes Parameter-Efficient Fine-Tuning (PEFT) techniques like LoRA and KubeRay's autoscaling capabilities to enable cost-efficient, on-demand GPU resource allocation for both research and production environments. Gain insights into how Ray is leveraged at various scales to create a flexible deployment solution, making full-stack development as accessible as full-scale production in Workday's multi-tenant ecosystem.

How Workday Fine-Tunes LLMs at Scale | Ray Summit 2024