Red Teaming Language Model Detectors with Language Models

Explore a 48-minute conference talk presented on 2/22/2024 by Yihan Wang from UCLA at the USC Information Sciences Institute. Delve into the investigation of robustness and reliability of large language model (LLM) detectors under adversarial attacks. Learn about two attack strategies: replacing words with context-appropriate synonyms and using instructional prompts to alter writing style. Understand the challenging setting where an auxiliary LLM, also protected by a detector, is used to generate word replacements or instructional prompts. Discover how these attacks effectively compromise detector performance, highlighting the urgent need for improved robustness in LLM-generated text detection systems. Gain insights into other recent works on trustworthy and ethical LLMs. The speaker, Yihan Wang, is a PhD candidate at UCLA focusing on trustworthy and generalizable machine learning, and a recipient of the 2023 UCLA-Amazon Fellowship.

Red Teaming Language Model Detectors with Language Models