Running Open Large Language Models in Production with Ollama and Serverless GPUs

Explore the deployment of open large language models in production environments using Ollama and serverless GPUs. Learn why companies are increasingly interested in running open models like Gemma and Llama, which offer full control over deployment, model upgrades, and data privacy. Discover how to leverage Ollama, a popular open-source LLM inference server, for both local and containerized environments. Gain practical insights into deploying an application that utilizes an open model with Ollama on Cloud Run, featuring scale-to-zero capabilities and serverless GPUs. This 43-minute talk from Devoxx provides valuable knowledge for organizations looking to harness the power of open LLMs while maintaining control over their AI infrastructure.

Running open large language models in production with Ollama and serverless GPUs by Wietse Venema