What's in My AI? A Comprehensive Analysis of Datasets Used to Train GPT 1, GPT 2, GPT 3, GPT NeoX 20

Explore a comprehensive analysis of datasets used to train major language models in this 53-minute conference talk from Devoxx. Delve into the evolution of pre-trained transformer language models from GPT-1 to Gopher, examining their potential as stepping stones towards artificial general intelligence (AGI). Investigate the lack of documentation for basic metrics such as dataset size, token count, and content details despite proposed standards for dataset composition and collection. Gain insights into the research covering 2018 to early 2022, synthesizing information on all datasets, including major components like Wikipedia and Common Crawl. Learn from world expert Dr. Alan D. Thompson as he shares his expertise in artificial intelligence, human intelligence augmentation, and the advancement of 'integrated AI'.

What's in my AI? A Comprehensive Analysis of Datasets Used to Train GPT 1, GPT 2, GPT 3, GPT NeoX 20