Unleashing Code Completion with LLMs - Learning from Building Code Suggestions

Explore the intricacies of building code suggestion tools using Large Language Models (LLMs) in this insightful conference talk by Monmayuri Ray. Delve into the learning journey of developing Code Suggestions, covering crucial aspects such as Model selection, ML Infrastructure, Evaluation methods, Compute requirements, and Cost considerations. Gain valuable insights from Ray's experience as an Engineering Manager specializing in AI-Assisted and MLOps at Gitlab. Learn about the essence of AI as "low-cost prediction" and DevOps as "low-cost transaction," and understand the importance of interdisciplinary collaboration in unlocking the potential of emerging technologies. Discover the key components of code completion tools, strategies for making LLMs useful, choosing the right roles, LLM architecture, and evaluation techniques. Whether you're a developer, data scientist, or AI enthusiast, this talk offers a comprehensive overview of leveraging LLMs for efficient code completion in production environments.

Introduction
Overview
What are Code Completion Tools
How to make LLMs useful
Choosing the right role
LLM Architecture
Evaluation
Recap
Outro