Test Time Compute: Verifiers and Parallel Sampling - Part 2

Dive into an advanced tutorial on test time compute and verifiers in large language models. Explore parallel sampling techniques, various verifier methods, and their implementation using vLLM with guided decoding. Learn how to optimize prompts for verifiers and compare different approaches like voting, scoring, and binary verifiers. Analyze results using Llama 3.2 1B model and review recent literature on verification methods in LLMs. Gain practical insights through hands-on demonstrations and access to comprehensive resources for further learning and experimentation.

Sampling and Verification
Training Compute vs Test Time Compute
Part 1 Recap: Sampling and Chain of Thought
Video Overview: Parallel Sampling and Filtering with Verifiers
How to sample multiple answers in parallel
Verifier Methods
Improving verifiers with fine-tuning or prompt optimisation
Output verifiers versus process verifiers
Majority Voting and Monte Carlo MCTS
Notebook Setup - Trelis.com/advanced-inference
Installation of vLLM with guided decoding
Loading Llama 3.2 1B as opposed to 3B in Part 1
Baseline Single-shot approach
Parallel sampling approach Pass@n / perfect verifier
Parallel sampling with a voting verifier using vLLM guided decoding
Prompt optimisation for verifiers
Parallel sampling with a scoring verifier 1-10
Parallel sampling with a binary true/false scoring verifier
Llama 3.2 1B Results
Literature Review Let’s Verify Step by Step, Large Language Monkeys, Are more LLM calls all you need? Tree of Thought
Resources