The Perl and Raku Conference 2024 in Las Vegas

Large Language Models (LLMs) have shown potential across a wide range of applications, yet their adoption, particularly in enterprise settings, has not kept pace with the general enthusiasm. In this paper, we present a novel application of LLMs to convert unstructured text into a structured data format, specifically the Resource Description Framework (RDF)—the lingua franca of the Semantic Web. We demonstrate how, by leveraging the structure of an existing RDF or OWL graph, we can automate prompts which allow for automated construction of Semantic Knowledge Graphs (SKGs).

Microcontrollers (μC) are single-chip integrated circuits which contain one or more CPUs along with memory and programmable input / output peripherals. Microcontrollers are used for embedded applications, such as industrial control systems and home appliances, some of which may be referred to as part of the Internet-of-Things (IoT). They are often a set of one or more interconnected circuit boards which can perform multiple tasks such as collecting and processing sensor data. For example, a microcontroller may process, communicate, monitor, and control hardware equipment such as motors, pumps, power generators, wind turbines, or automotive engines. Billions of microcontroller chips are manufactured and sold by various semiconductor companies every year, along with packaged software so their customers can program the devices to suit their needs.

C language cross-compilers are widely used to generate assembly (ASM) code for a specific μC device. Perl interpreters & cross-compilers that support a subset of the Perl syntax could be implemented to program, monitor, and debug microcontrollers. Such a Perl cross-compiler would run on a desktop Perl environment and generate a binary image which can be downloaded onto the microcontroller's circuit board.

A cross-compiler ecosystem would be mutually beneficial for both Perl developers and end users who rely on microcontroller-based hardware systems. This would open up opportunities to re-use Perl libraries which currently run only on normal desktop and server CPUs. Expertise in programming IoT hardware via Perl would not only create jobs for college graduates who spend time and resources to learn Perl, but also help existing Perl developers to improve their career options and pay grades, because many embedded-systems jobs demand a great deal of domain-specific knowledge and thus offer higher salaries.

The ADCIRC Surge Guidance System is a portable real-time operational storm surge forecasting framework, forged over 15 Atlantic hurricane seasons, that is used to deliver critical information to emergency managers on federal, state, and local levels in Louisiana, Texas, and North Carolina and within groups such FEMA, NOAA, and DHS. Over the years it has saved millions of dollars in time, property, and emergency assets. It has also likely saved many lives.

This paper discusses its humble origins from a collaboration between LSU and UNC in the early wake of Hurricane Katrina (2005), to its first real test during Hurricane Gustav (2008) and successes over the years. Much of this paper discusses the technical aspects of the system, how the user experience has been tailored for real-time operations, and the technical decisions that have been made leading directly to its success as a robust and adaptable framework.

Most relevant for the intended audience is how using Perl, Bash shell scripting, and standard Unix tools has made all of this possible.

Component-Based Software Engineering (CBSE) assembles pre-existing, reusable software components into new applications. CBSE targeting scientific applications (SCBSE) has the potential to instill novel functionalities in existing, extensively tested components with the promise of rapid development cycles that are particularly relevant for fast moving, data-intensive fields such as bioinformatics.

Dynamically-typed scripting languages have played a particularly crucial role in SCBSE by acting as glue that integrates and coordinates heterogeneous components, or by facilitating communication of components that are organized as filters in complex data flows. Such languages confer additional benefits including rapid prototyping and support for multiple programming paradigms, thus allowing effortless exploration of multiple architectural alternatives before settling on the final design.

Perl has traditionally been the go-to dynamically-typed scripting language in bioinformatics (“BioPerl”) due to its robust text manipulation capabilities, but its declining overall popularity has also affected its standing in this field. The recent under-utilization of Perl in bioinformatics represents a significant missed opportunity to enhance applications by leveraging the vast resources in the Comprehensive Perl Archive Network (CPAN), a versatile and rich choice of Object-Oriented (OO) modules and a mature framework (Alien) for making external libraries and tools available to Perl for component-based application building. Such applications can also leverage Perl’s maturing framework for interacting with libraries in non-Perl languages using Foreign Function Interfaces (FFI) and the Perl Data Language (PDL), thus providing additional memory-based options to the traditional filter-based communication scheme that has been the mainstream approach in bioinformatics.

To the extent that these external applications and libraries can support the OpenMP “fork-join” multi-threading paradigm, the resulting component-based Perl application will be endowed with both coarse-grained (process-based) and fine-grained (thread-based) parallelism to adapt to the properties of the hardware environment.

In this paper we illustrate the value of Perl for SCBSE in bioinformatics by combining OO Perl, Alien, PDL, FFI and OpenMP in order to enhance two bioinformatic applications: 1) the R-based RNA-sequencing simulator “polyester”, and 2) the biological sequence similarity database search tool “edlib”. We utilize lightweight OO schemas and PDL to enhance the first tool, endowing it with capabilities to simulate additional processes (tailing with long poly-adenine, polyA, tail) which operate during the generation of messenger RNA. We then use this application to develop a novel, native Perl approach to trimming these tails, based on regular expressions, and provide a fast alternative to the Python application “cutadapt” which has been the gold standard in the field for years.

Our enhancement of “edlib”, which is a single-threaded command-line application and library, proceeds along an entirely different pathway: the introduction of coarse-level parallelism through the Many Core Engine (MCE) for the application, and OpenMP for the underlying library through the Platypus::FFI framework. These approaches used in combination can provide customizable levels of coarse- and fine-grained parallelism for the data- and compute-intensive task of sequence analysis, and provide proof-of-concept for the utility of the Bio::SeqAlignment framework currently under development in Perl.

This paper presents the tree editor TrEd and related tools that can be used to create, modify, browse, and search treebanks - large language corpora annotated with syntactic and/or semantic structure information. This might include not only phrase structure or dependencies, but also co-reference, discourse analysis, and even inter-sentence relations.

The project started in the year 2000, and it has been in continuous use since then at various institutions all over the world. Most of the tools are written in Perl, which makes them available to all major operating systems.

As a by-product of the treebank creation, various aspects of the annotation process have been studied, e.g. inter-annotator agreement, or the influence of automatic pre-annotation on the speed and accuracy of the annotation.

For searching the treebanks, a query language was developed that describes sets of tree nodes and the relations between them. It also supports aggregation to produce quantitative outputs. There are two different implementations: one translates the queries into SQL statements, and the other searches the data directly in the editor.

Originally, TrEd supported the PML data format used for the Prague Dependency Treebank. To process data in a different format, one first needed to convert the data into the PML format, and then possibly convert the modified data back to the initial format. Later, a versatile extension system was added to TrEd which made it possible to support other data formats directly.

We will show how this works on the example of Universal Dependencies (UD), which is a framework for grammar annotation across different human languages. It has over 500 contributors, who have so far produced more than 200 treebanks in over 100 languages. This extension allows TrEd (and some other tools) to open the files in the original UD format natively, building the internal representation on the fly, and also serialize them back after editing.

PerlGPT is a large language model (LLM) comparable to ChatGPT 4.0, and trained on additional Perl-related content only. PerlGPT will be based on Meta's Code Llama (from Llama v2 or newer) language models, with all new components implemented in Perl where possible and released as free-and-open-source software (FOSS), unlike ChatGPT and other proprietary LLM systems.

Code Llama: Open Foundation Models for Code

Code Llama's documentation currently lists the following primarily-supported languages: "P****n, C++, Java, PHP, TypeScript, C#, and Bash". One of the outcomes of the PerlGPT project will be to effectively add Perl to this list. (PerlGPT may ultimately be released under an alternative name, such as "Perllama".)

Phase 1 consists of training a 13B input language model using Perl-related stimulus/response pairs curated from CPAN, and potentially other public Perl-specific data sources. The goal of phase 1 will be to deliver an LLM capable of generating pure-Perl source code in collaboration with a Perl programmer, similar to Microsoft Bing and GitHub Copilot. We will demonstrate our most up-to-date version of PerlGPT during the live conference.

The extensive functionality of the Perl language and its clear, C-like syntax make it a good choice for scientific data analysis. This work demonstrates the use of Perl in analytical chemistry, specifically near infrared (NIR) spectroscopy, a non-destructive, non-invasive, rapid method of chemical analysis. Example applications of Perl for pharmaceutical analysis are reported. A portable, handheld NIR spectrometer (model NIR-S-G1, InnoSpectra Corporation) is controlled by a graphical user-interface (GUI) written in Perl/Tk. Resulting spectra are simultaneously interpreted with results reported instantaneously to the end-user.

NIR absorptions arise due to overtone vibrations of chemical bonds that are approximate multiples of the fundamental mid-infrared absorption frequencies (i.e. first, second and third overtones) or are combination bands of several fundamental vibrations’ frequencies. Reduction in reflected incident light intensity by a material at a given wavelength, relative to that from a non-absorbing, diffusely-reflecting standard (e.g. sintered PTFE) occurs due to absorption by one or more covalent bonds of the material studied. Bonds that absorb in this region of the electromagnetic spectrum generally involve hydrogen covalently bonded to either: carbon, oxygen or nitrogen. Most organic molecules, including pharmaceutical active ingredients, thus show absorptions in this region.

An NIR spectrum of a chemical-sample is a multivariate data array of N-reflected intensities of light at discrete wavelengths (i.e. 1-by-N) across the wavelength range (approximately: 900 – 1,700 nm). With many variables (wavelengths) in a spectrum (between one and several hundreds of wavelengths), interpretation of a spectrum or a collection of spectra requires multivariate analysis (termed chemometrics).

This work illustrates the use of Perl twofold. The first application is for control of the open-source scientific computing programming language GNU Octave, to generate multiple NIR data calibration models for quantification of analyte (pharmaceutical active ingredient). These multivariate-regression models are produced by calibration of NIR data with reference analytical method concentration data, using the partial least squares regression (PLSR) algorithm. Various mathematical data pretreatments are applied to the NIR spectra and, in combination, subsets of wavelength regions are also utilised in the data modelling. The combined effects of these on the accuracy and precision of calibration is determined. The aim of this data modelling is to select a model with high accuracy and precision.

The second application of Perl is for control of the NIR spectrometer. A Perl/Tk developed GUI is used to control the handheld NIR spectrometer, enabling acquisition of: background spectra of the reference reflectance standard (for instrument response calibration) and then of spectra of chemical samples. The GUI enables interpretation of the acquired spectrum on-the-fly, for a qualitative (i.e. identification) or quantitative chemometric model. Perl scripts load ASCII text arrays – previously developed, optimised GNU Octave chemometric model and chemical-sample spectrum – and perform multivariate analysis. The GUI then reports to the end-user the required metric for either identification (along with a pass/fail result) or analyte quantification. This therefore obviates the need for the end-user to perform chemometrics.

The presentation will include illustration of NIR analysis for one or more pharmaceutical applications.

While some tools exist to reason about the correctness of a program and its fitness for particular purposes, there remain a number of unexamined approaches which will likely prove fruitful to explore.

The productivity of any piece of capital or laborer is entirely a function of its error rate; being able to quantify the conditions under which failures can be expected is thus of the utmost importance.

In the era of testsuite generation via large language models, this is of even greater concern.

I will lay out what is currently available, how that fits into a framework for reasoning about programs in a rigorous fashion, and identify places where our ability to reason could be improved via tooling.

Prototypes of said tools will also be demonstrated in Perl.

A discussion of some do's and don'ts while trying to balance community building and code quality in a distributed open source product using the MoSCoW prioritization technique as a framework for QA.When performing QA on code submissions in an open source project, you may be interacting with new members to the community, and people you have never met. While maintaining the guidelines of the community and ensuring code quality are top concerns, a nod must be given to building and uplifting the community, even when failing QA. The MoSCoW method for prioritization, often used in project planning, provides a good framework for giving the necessary feedback, making clear what is required, and giving space for opinion.Audience: All["perl", "QA", "Koha", "Community"]

There are many existing PDF modules for Perl, most of which use high-level API calls and hide the PDF internals. PDF::Data takes a different approach, directly representing low-level PDF internals as Perl data structures which can be freely manipulated, and then converted back to PDF format.Explore the capabilities of PDF::Data, a Perl module that offers a unique approach to PDF manipulation by directly exposing PDF internals as modifiable Perl data structures. Unlike traditional PDF tools which shield users from the complexities of PDF internals with high-level APIs, PDF::Data allows for detailed, low-level interactions. This talk will delve into how developers can leverage PDF::Data to gain unprecedented control over PDF content, enabling precise customizations and manipulations that are not possible with standard PDF libraries. We will cover practical examples including dynamic document generation, content extraction, and direct modifications to the PDF structure, providing a toolkit for advanced PDF solutions in Perl.Audience: Advanced["Perl", "Data", "Open Source"]

Your project's README stinks. Let's make it better, together!Your project's README on Github is your project's landing page, your one chance to land a solid first impression with a new user. So why are so many READMEs so boring? With minimal effort, you can make a lasting impression that win over users who might otherwise move on to another project instead.

Jason Crome will show you some techniques for making an eye-popping README for your project, using Request Tracker, Dancer, and other popular projects as models for what can be done. By the end of this short talk, you'll be able to take your project's README to the next level and set yourself apart from other open source projects.Audience: All["Perl", "Raku", "Open Source", "Community"]

Want to extend Perl with libraries written in other languages? Interested in adding a secure plugin system to your Perl app? Want to stay away from building and linking native extensions? This talk will cover extending Perl with Extism, a lightweight framework for building with WebAssembly.Traditionally, Perl is extended with libraries written in other languages using the C ABI and linking via an XS extension or FFI. Building and linking a native library or extension into Perl can be error prone and even introduce memory safety issues into a Perl program. Building to WebAssembly (Wasm) enables running code from a multitude of programming languages without building, running, and linking them as native code. How can we easily integrate running Wasm with Perl to allow extending Perl without perils of linking native code?

In this talk, I’ll start by providing basic background information on extending Perl and discussing why you might want to use Wasm instead of linking native extensions. Then, I’ll contrast the development workflow using Wasm both directly and with the [Extism](https://extism.org/) framework. Finally, I’ll showcase how Extism can be used to implement a sandboxed plugin system in your Perl app that supports a multitude of programming languages.Audience: All["Perl", "Open Source", "Native"]

Actually Portable Perl (APPerl) is a distribution of Perl that runs on several OSs via the same binary. APPerl builds to a single binary with Perl modules packed inside of it. This talk will cover how APPerl works and how it can be used to build cross-platform, single binary, standalone Perl apps.Programming in Perl is awesome, but how can you get non-Perl programmers, even non-technical users to run your program? The easiest way might be to ship a binary. Tools such as [PAR::Packer](https://metacpan.org/pod/PAR::Packer) can be a great way to create a binary, but it requires packing on each target system and is difficult to do right on non-Windows platforms (often due to dependence on system libc). The [Cosmopolitan Libc](https://github.com/jart/cosmopolitan#cosmopolitan) enables creating polygot executables that run on multiple operating systems including Windows, Linux, Mac, and more with the same binary! [APPerl](https://computoid.com/APPerl/) is a distribution of Perl built with the Cosmopolitan Libc that enables creating cross-platform, single binary, standalone Perl apps.

In this talk, I'll start by briefly going over the challenges distributing Perl applications. Then, I'll briefly explain what Actually Portable Executables are, show how they work, and how Perl can be built as one. Finally, I'll cover the leap between being a specialized Perl Distribution (APPerl) and working as a binary packager and demonstrate how APPerl can be used to package up Perl applications.Audience: All["Perl", "Open Source", "Polyglot", "Native"]

How do you create music with perl, from scratch? Drop-in to find out how, with details, examples, and audio too, of course! This will be a short presentation, given the wide scope of the topic. I will cover the basic concepts of algorithmic music, introduce available CPAN modules, show how to use a few, including when and why to use them. Audience: All["Perl", "Algorithmic", "MIDI", "Music", "Beginner Talk"]

Phrasing and voicing techniques with example algorithms and audio! Also: How to interactivly respond to events with MIDI!
This presentation will introduce fundamental phrasing and voicing techniques, and "real time MIDI" too. Which modules to use will be discussed, of course.
Audience: Intermediate ["Perl", "Open Source", "Fun", "Music", "Intermediate", "MIDI"]

Did you know that Perl has 5 new AI projects?

- OpenAI ChatGPT
- TensorFlow Neural Networks
- Cowl Ontologies
- PerlGPT Large Language Models
- ChatGPU Navi™ AI

Artificial intelligence, machine learning, and the future of Perl... don't miss it! Future-proof your career with Perl AI.In this talk, we'll review the current status of all major AI projects in Perl.

We will also attempt to answer a number of frequently asked questions, such as:

- Do I have to pay to use ChatGPT in Perl?
- What is a neural network?
- What is an ontology?
- Can AI write Perl source code?
- Can we create strong AI in Perl?

Artificial intelligence is the way of the future, and is Perl's big chance to make a comeback.

Let's make the most of this AI opportunity to join together and put Perl back on top!Audience: All["perl", "ai", "artificialintelligence", "aiperlcommittee", "ml", "machinelearning", "fun", "community", "future", "Perl", "Open Source", "Fun", "Community"]

exercism.org is an online open-source platform for people to complete syllabuses, exercises, and receive mentoring in a variety of different programming languages. In my talk I will describe the development of the Perl and Raku tracks and how I hope to encourage more people to learn these languages.This talk will go in detail about how Perl and Raku have been integrated on exercism.org. I will describe how exercises have been implemented for our languages, how the in-browser editor runs Perl and Raku tests, the current state of the work-in-progress syllabuses for these languages, and how I try to gently introduce people coming from other programming languages into Perl and Raku.Audience: Beginner["Raku", "Perl", "Open Source", "Fun", "Polyglot", "Community"]