The Perl and Raku Conference 2024 in Las Vegas

ETL (Extract, Transform, and Load) is about getting data from there to here. In today’s world that means handling lots of data, usually in parallel, usually with tight timelines. Raku’s parallel make and lazy gather form the basics of a nice pattern for general ETL, separating out low-level tools for extracting and munging the input data from the higher-level constructs for actually processing it. This talk looks at some fundamental ETL input varieties – lines, chunks – and how to use Raku as the glue for managing their import via flat files or data channels.

This talk expands on the older talk ‘Hypers & Gathers & Takes, Oh my!’, going deeper into parallel make and lazy gather.