Once Upon a File - Navigating Data Import and Export Challenges

Explore common challenges and amusing anecdotes related to file handling and data import/export in this 28-minute conference talk from Dyalog '23. Delve into the intricacies of various file formats, including plain text, fixed-width, CSV, XML, and Microsoft Excel. Learn about operating system-specific line endings, quotation mark issues, UTF-8 encoding, and the impact of regional settings on datetime formats. Discover strategies for dealing with formatted cells, using Excel programs for file operations, and coping with unexpected problems. Gain insights from Elena Paviotti and Gianna Folladore's experiences at SimCorp Sofia to improve your approach to file management and data processing.

About SimCorp Sofia
Problems dealing with data import and export
Common formats used in Sofia
Issues with plain text files
Operating system-specific line endings
Tables in fixed width text files
Fixed width delimited file
Comma Separated Values CSV
Quotation marks
Confusing CSV and Microsoft Excel files
UTF-8, ⎕NREAD, ⎕NGET and BOM
XML
Microsoft Excel
Formatted cells
Regional settings and datetimes
Using Excel program to read and write files
How to cope when things go wrong