Solving Real-World Data Science Problems with LLMs - Historical Document Analysis

Dive into a comprehensive video tutorial on analyzing historical documents using Python and Large Language Models (LLMs). Learn to set up LLMs with both OpenAI API and open-source Llama 2 via Ollama, parse entities from text, and work with real-world data from the Freedmen's Bureau historical documents. Follow along as the instructor demonstrates connecting pages from the same documents, extracting key information like names, ages, and locations, and analyzing the resulting entities. Gain practical experience in data science and natural language processing while uncovering insights from historical records. Perfect for those interested in applying AI techniques to historical research and document analysis.

- Video Overview & Reference Material
- Data & Code Setup
- Task #0: Configure LLM to use with Python OpenAI API
- Task #0 continued: LLM Configuration with Open-Source Model LLama 2 via Ollama
- Task #1: Use LLM to Parse Simple Sentence Examples
- Sub-task #1: Convert string to Python Object
- Task #1 continued: Use Open-Source LLM to Parse Sentence Examples w/ LangChain
- Quick note on a benefit of using LangChain easily switching between models
- Task #2 warmup: Grab Apprenticeship Agreement rows from Dataframe
- Task #2: Connect Pages that Belong to the Same Documents
- Task #3: Parse out values from merged documents
- Task #4 setup: Analyze Results
- Fixing up our results from task #3 quickly
- Task #4: Find the average age of apprentices in our merged contract documents
- Other analysis, wlho had the most apprentices?