Six reasons to try out Perl 6

I hate list articles, but sometimes they're a necessary evil. We humans are more inclined to enumerated lists of information rather than open-ended discussion, I suspect, because we feel like there is less investment. When we see "five reasons to jog," we assume that we'll get a reason right off the bat that we can evaluate and just stop reading if we don't agree.

But with an essay or other generic prose format, it can be difficult to tell just when the thesis will be delivered, and therefore there is more cognitive load in investigating the content.

To that end, I thought I would provide a simple, easily read (and probably easily dismissed) list of the reasons that Perl 6 is a programming language that you should be exploring.

Before we start, though, let's determine what constitutes a valid reason to learn a new programming language. Certainly necessity is quite common. I learned BASIC in the 1980s because it was the only language I had available to me on my Tandy Color Computer (dubbed "CoCo" by its users). I didn't chose BASIC, it chose me. Today, this is commonly the case when people take a job or attend a course which uses a particular language. Perl 6 is obviously too new a language to impose itself on people in this way, so that's certainly not on the list.

If you are not forced to use a particular language, then it comes down to utility, but utility functions are complicated beasts, especially when talking about ways of expressing ideas. As an excellent example, let's look at Perl's (and many of its predecessor languages') use of a prefix (or suffix) character to indicate a data type. In Perl this has traditionally been a dollar sign for scalar (singular) values, at-sign for arrays and percent for hashes (sometimes called dictionaries or more technically unordered associative lists in other languages). This affords Perl a great deal of flexibility. Variables can be called out in strings (e.g. "I am $name") and there is no such thing as a reserved word for variables (e.g. $while is perfectly valid). It also gives someone reading code the opportunity to quickly scan for variable usage and visually pattern-match the "shape" of the code.

But to non-Perl programmers, this feature often induces frustration. It's harder to read code initially when you're not used to this convention. So, does this make Perl a good or bad language to learn?

For the sake of this article, I'm going to say that it has no impact. I'm only interested in language features which will either improve the quality of code or the capabilities of the programmer. For example, learning any Algol-derived language from C to Perl to Java, etc. will teach a programmer to understand code in any language that mixes procedural function calling with mathematically-inspired infix operators. This is an important skill in the world of modern programming languages, and it will improve the ability of the programmer to learn nearly any modern language. Getting used to variables with prefixes won't really change the way you code or enable you to learn other languages, so it wouldn't be nearly as useful as this.

Okay, so on to the list...

Number 6: Grammars

While many people tout Perl 6 grammars as the primary innovation of the language, I'm going to place it at number six, here, because it's not really the most fundamental shift you will experience in the language. Still, it needs to be on this list. Just about every language has access to a parser generator, but Perl 6 places the parser directly into the language as a first-class feature. In fact, the language uses its own grammar feature to parse itself!

Essentially this is the ability to define a parser much the way you would define a class. Grammars can derive from each other like classes, and they can even have attributes much as a class would (this is because, under the hood, they use the same infrastructure in the language as a class).

One thing that I think this will improve is the reliability of programs that need to access user-provided data. All too often, user data is handled with poorly crafted regular expressions that either fail to express the possible complexity of user input or don't sufficiently constrain it due to the limitations of regexes.

Number 5: Modern OO

In most modern OO languages, features such as a metaobject protocol, automatic initializers and accessors and the like are commonplace. This is certainly the case with Perl 6, but the language goes far beyond this. Trait-like roles that can be generics, monkey-patching objects, indirect method invocation, redispatch, and dozens of other advanced features are combined in Perl 6's object system making it a joy to use in a diverse array of use-cases.

Number 4: Laziness

If you are used to languages like CommonLisp or Haskell, you will know the power of lazy evaluation being the default. Unlike languages that have token access to generators and iterators, Perl 6 and the other languages I mentioned, provide laziness as a primary mode of accessing sequential data. It's almost always the case that laziness is involved in any such transaction and often in a way that is transparent to the user. For example, constructing a range from a variable start point to a variable end point is fairly common in most languages, but you usually have to use an alternate mechanism is either of those endpoints are going to be infinite. In Perl 6, there is no such distinction. Ranges behave the same no matter what their endpoints might be.

Number 3: Deep Gradual Typing

It's possible to construct a function in Perl 6 which takes a scalar variable and does something with it, regardless of its type. In fact, that's the default. However, if you want to specify a type for a parameter, you can. The type of an input parameter is checked and if it matches, you're good to go.

But that's only the start. The "where" operator allows the programmer to further constrain a parameter in a dynamic way (e.g. requiring that a numeric parameter falls within a specific range) and the "subset" operator allows the declaration of a type-like name that includes such parameterization. This allows a mix of static and duck typing in a convenient syntax.

This typing goes further, allowing constraints on the contents of sequential and associative data types, class interface assertions (through "roles") and many other advanced type management features.

Number 2:  Slangs

Slangs are variations on Perl 6's parser. They can be defined at runtime in Perl 6 and can be used lexically like a variable definition. A slang allows anything from a trivial feature change (perhaps you want a conditional block that used the keyword "maybe") to more substantial additions (say, adding a quoting operator for JSON data that validates the data at compile-time) to fundamental replacements of the parser (e.g. to lexically scope a block of Perl 5 code).

Slangs offer the ultimate in language flexibility, and because the underlying runtime is so flexible and full-featured, nearly anything you might want to do is possible. Want to define a Python object with Python's method call semantics? It's just a matter of getting in there and writing the code!

Number 1: Access to Perl 5 and C

Through the "Inline::" modules, Perl 6 has access to Perl 5 and C code. This immediately gives the language access to two of the largest suites of libraries in the world. Essentially, from day 1, Perl 6 has more library support than any language I have ever seen as this stage of development.

While new features are all well and good, one of the crippling problems with many new languages is that they have very few features available to them through library code. This is why the JVM is such a popular platform for new languages, but constraining implementation to one VM is hardly the right solution to such problems.

Honorable mentions

There are too many great features in Perl 6 to mention in one document, but a few that didn't quite make the list are:

• Automatic use of rationals for integer division
• The most advanced Unicode handling I have ever seen
• True concurrency without a GIL

Perl 6: What's that you've got there? On stringification.

In Perl 5 and Python, the two languages I'm most familiar with outside of Perl 6, at this point, strings are pretty simple things. When you have an object and you want to print it, for example, here's the Perl 5:

    say $thing;

And here's the Python:

    print thing

Notice that there's no hinting being given to the compiler as to what this "thing" is or how to render it to the output. In Perl 5, this is an implicit feature of all scalar values. They have a string representation which can always be fetched. It might not be useful (and in many cases isn't) but it'll be there.

In Python, print automatically converts its parameters to strings as if you had written:

    print str(thing)

though it can be more complex than that, due to encoding issues. The default stringification for objects is the __repr__ method on the object's class, which is (as in Perl 5) not terribly useful, but good object authors know to override it and/or the stringification method (__str__) with something more useful.

In Perl 6, the picture is both more complicated and more functional out of the box.

There are three kinds of stringification:

1. Coercing to str (usually via the ~ unary or binary operator)
2. The .gist method
3. The .perl method

The first is a straightforward "what are you as a string"? That has no implication of preserving all aspects of an object. So, for example, when matching a regular expression and getting back a match object, you might want to print the whole match. This is done by coercing the match to a string:

    given "fool" {
        say ~ m/fo+/
    }

Will print "foo". Notice that there's quite a lot of information thrown away by this operation. There's the position in the base string, any sub-matches, etc. all available in the match object, but when stringified, that's all thrown out the window, and the "most salient" string elements are printed.

The second form of stringification is called a "gist" and it's accessed by using the .gist method. Here's an example:

    given "fool" {
        say m/f(o+)/.gist
    }

prints:

｢foo｣
 0 => ｢oo｣

In a gist, the stringified version of an object tries to strike a balance between preserving internal information necessary for debugging or other analysis and readable text. The general rule appears to be:

• Throw away internal state that's not relevant to the textual understanding of the object (e.g. the "from" and "to" attributes of a regular expression match
• Use the corner-brackets (「 」) from Asian scripts to enclose internal string value(s)
• Recursively descend data structures, calling each of their gist methods

Finally there's the third option, which is exactly like Perl 5's Data::Dumper and a bit like Python's repr except that it aggressively attempts to determine how to represent code as executable Perl. Unless there's ephemeral state involved, evaluating the output of the .perl method on an object should yield a copy of the object.

Languages, in general are moving towards this sort of "structured intuition" about the representation of objects, rather than some more on-demand way of representing data. It allows authors to carefully control how their objects are represented, but also gives simple objects all of the tools that they need out of the box.

Using my stardock image on Amazon AWS free tier

Amazon's AWS service is great. I love the fact that it provides a free server to develop on and if I ever need more, I can pay for it.

Just one catch: many of the things I enjoy working with are small enough to fit in RAM to run, but not to build. Perl 6 is a great example. See my previous posting on building a "stardock" container for Perl 6. Anyway, that won't build on Amazon's AWS free tier, so what do I do? Well, I can spin up a local VM under a Windows box to do the build, so here's what I do:

docker build --tag=stardock .
docker save -o stardock.img stardock
scp stardock.img <awshost>:.

And then on the target system:

docker load -i stardock.img
docker run -it stardock perl6 -v

Now you have a build of Perl 6 Rakudo Star from the most recent HEAD, but without having to have enough RAM in your free dev environment on Amazon!

Building bleeding edge Perl 6 as a Docker container

I got tired of re-building Perl 6 on every system I touch, but the public Rakudo Star build of Perl 6 that's available as a Docker image is pretty badly out of date. If you want to get and build the latest and greatest, here's how to do it in a container.

First, create your "Dockerfile" with this as its body:

FROM ubuntu:15.04

RUN apt-get update && apt-get -y upgrade && apt-get clean
RUN apt-get -y install perl && apt-get clean
RUN apt-get -y install git-core && apt-get clean
run apt-get -y install build-essential && apt-get clean

RUN git clone https://github.com/rakudo/star.git
RUN cd star && perl -i.bak -pE \
  's/git\@github.com\:/git\:\/\/github.com\//g' .gitmodules
RUN cd star && git submodule sync && git submodule init && git submodule update
RUN cd star && git clone https://github.com/rakudo/rakudo.git
RUN cd star && perl Configure.pl \
  --force --prefix=/usr --backends=moar --gen-moar --gen-nqp
RUN cd star && make && make install

CMD [ "shell" ]

Now build your image by running docker build:

$ docker build --tag=stardock .

I call my image stardock. You can call yours whatever you like. Note that the build is too large to fit in some minimal environments. It may fail, for example, on a free tier AWS EC2 instance.

Next, you want to test your build:

$ docker run -it stardock
root@5bf4853308cf:/# perl6 -v
This is perl6 version 2015.06-225-g3bdd0af built on MoarVM version 2015.06-88-g647df11
root@5bf4853308cf:/# perl6 -e 'say "Hello, world!"'
Hello, world!

And there you go, a working Perl 6 built off the latest rev of Rakudo build per the Rakudo-Star release packaging.

Because the point of this is to be up-to-date, you might want to add this line before the "RUN git clone" of Rakudo:

ADD update.txt update.txt

and just "touch update.txt" in your Dockerfile directory before you do the build to force it to pick up the latest Rakudo and rebuild from scratch.

Sand: Rules

Sand is a programming language that I introduced in a previous article. Sand rules are not completely fleshed out, but here are the primary design goals:

• To be as similar to perl 6 rules as possible
• To reduce complexity where it does not yield substantial benefit and where reduction in complexity does not come at tremendous cost to compatibility
• Speed of parsing and execution

Specifically, these are the top-level elements:

• A regex is a sequence of atomic assertions about patterns very similar to Perl 5 regular expressions.
• An assertion is embedded code within a regex which returns true or false and matches on true.
• A subexpression is a reference from one named regex to another.
• A token is a regex which matches in a single pass (no backtracking) or fails.
• A rule is a regex which defaults to considering whitespace significant.
• Tokens, rules and bare regexes are all optionally named, allowing subexpressions to use those names for reference.
• All of the above may be referred to as elements of a grammar and are technically methods. They must either be declared as part of a class that implements (does) the role, "grammar" or must be implicitly associated with the global "_regex" grammar (e.g. this code defines a method on _regex implicitly and invokes it: if "foo" ~~ regex{f} {...})

here are some examples:

class Sand :does(grammar) {
    method same_category($letter, $category by reference) {
        my $letter_cat = $letter.unicode_block_category();
        if $category {
            assert($letter_cat == $category);
        } else {
            $category = $letter_cat;
        }
    }
    method is_token_unicode() -> (!bool) {
        my $number = undef;
        my $other = undef;
        for self.match(0).str().list() -> ($letter) {
            if $letter.isdigit() {
                self.same_category($letter, $number);
            } else {
                self.same_category($letter, $other);
            }
        }
        return True;
    }
    token identifier {
        [ <alpha> | '_' ] <alphanum>* { .is_token_unicode() }
    }
    rule scalar { '$'<identifier> }
}

This code defines the "identifier" and "scalar" regexes that are part of Sand's grammar. Notice that a grammar class can define methods like any other class and those methods can be invoked from within the body of a regex.

The Sand programming language

The following is a direct cut-and-paste from my specification of the Sand programming language on my Wiki (which is now down and may never be revived, depending on how much free time I have). This copy was fetched from Google's cache.

Python line continuation

PEP 8 is Python's Bible when it comes to style, but it deliberately avoids certain kinds of assertions about how to write code while suggesting some only in examples. One area in which I feel it is too vague is line-continuation. In this article, I'll try to outline the reasons that one wants to continue lines and how one should deal with that. This is partly a matter of preference, but most of the rationales I present will come along with the practical reasons that this choices make sense.

Let's start with the simplest case. You have a line where a simple function call extends past the end of the normal boundary (79 characters per PEP 8, though I find myself trying to stay below that for historical reasons relating to specific editors, formatters and printing systems):

do_the_first_things_first(thefirstthing, thesecondthing, thelastofthethings, reasons=None)

There are many ways that would work. Here's one:

do_the_first_things_first(thefirstthing,
                          thesecondthing,
                          thelastofthethings,
                          reasons=None)