Binary File Format Engineering and Reverse Engineering

Explore the intricacies of binary file formats and reverse engineering in this comprehensive conference talk from ACCU 2023. Delve into the evolution of computer data representation, from basic bits to complex multimedia files. Learn about file format conventions, variable length encoding, protobuf, compression techniques, and data transforms. Discover the pros and cons of creating binary files, various types of binary file structures, and their practical applications. Gain insights into working with binary data, including strategies for finding, using, and grouping information. Perfect for software engineers and enthusiasts interested in understanding the underlying mechanisms of digital file systems and reverse engineering techniques.

Intro
What are bits
Conventions
Variable Length
Protobuf
Compression
Encoding
Data transforms
Variable length codes
Bitpacking
Questions
Why not to make binary files
Types of binary files
Header and flat files
Index and data files
Record sequences
Chunked files
Use cases
Working sets
Logical Use Cases
Finding the Data
Using the Data
Grouping the Data